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ABSTRACT 

This study was performed in response to the 1989 
Department of Defense Appropriations Bill, which directed the 
Department of Defense to conduct a study on the use of interactive 
videodisk technology in education and training as it pertains to 
effectiveness, cost effectiveness, timo on task, retention, and 
overall applicability to current and future Department of Defense 
(DoD) training and education requirements. A quantitative meta 
analysis was performed of 31 evaluation studies concerned with 
interactive video instruction. Across the evaluation studies 
reviewed, interactive video instn^ction was found to be more 
effective than conventional approaches to instruction. The 
effectiveness was enhanced further by the utilizacion of more 
interactive features and directed rather than freeplay simulations. 
Use of interactive video can increase training efficiency by 
increasing opportunities for practice and requiring students to 
participate ipore actively in instruction. Three studies indicated 
that retention usr.ng interactive video instruction is at least equal 
to that found using other instructional approaches. Interactive video 
i3 a promising and cost-effective medium that will have a significant 
impact on military sys;:ems. An executive summary of a study of the 
effectiveness of interactive videodisk in army communications 
training and a list of organizations contacted are appended. (58 
references and 2 tables) (GL) 
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SUMMARY 



Background 

The challenges facing militaiy trainers are substantial and growing. Trainers have sought 
new app^aacnes in instructional technology to enable them to noeet these challenges. 
Interactive videodisc instruction requires students to participate actively in the training 
environment, it provides students accrss to high quality simulation of devices that could not 
odierwise be made available to them, and it distributes bodi the content and interactions of 
high quality training to widely dispersed sites. As a consequence, interactive videodisc 
instruction has been prominent among die new iy>proaches pursued by the Department of 
Defense, and Defense research and development laboratories have been leaders in 
developing new instructional approaches based on interactive videodisc technology. 

Purpose 

This study was perfcnned in response to the 1989 Department erf Defense Appropriations 
Bill which directed the Department of Defense to conduct a study on the use of interactive 
videodisc technology in education and training as it pertains to effectiveness, cost- 
effectiveness, time OD task, retention, and overall applicability to current and future DoD 
training and educaticm requirements. 

A quantitative, analytic review was performed of evaluation studies concerned with 
interactive videodisc instruction. It is the first "meta-analytic" review undertaken for 
videodisc instruction. Thirty-one studies were identified as appropriate for inclusion. Of 
these, 15 concerned military training, 3 concerned industrial training, and 13 concerned 
applications in higher education. 
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Findings 



Overall effect size obtained from this anal^ v%'as .51, which suggests that through the 
introduction of interactive videodisc instruction 50th percentile students might be expected 
to perform at what is currently the 70d) percentile. Different results weie obtained for 
different instructional settings. The average effect size obtained for military training was 
.35, suggesting a rise for 50th percentile students to the 5^ percentile. The average effect 
size obtained for industrial training was .30 suggesting a rise for SOth percentile students to 
the 62nd percentile. Hie average effect size obtained for higher education was .70, also 
suggesting a rise for 50th percentile students to Uie 76tii percentile. 

Interactive videodisc instruction has been demonstrated to be die preferred cost-effective 
alternative in twcral specific applications. Cost ratios for interactive videodisc applications 
over alternative approaches ranged from .02 to 4.49, suggesting wide variability. 

One study was found tiutt discussed time on task as a result of introducing interactive 
videodisc instniction - time on task in this study increased by 45 percent 

Superior performance by interactive videodisc groups was sustained over retention periods 
ranging frcxn one to ten weeks. However, performance of the interactive videodisc groups 
grew closer to that of groups using otfier instructional approaches over these retention 
intervals. 

Conclusions 

The following were the conclusions of this study: 
Effectiveness! 

1. Across die evaluation studies reviewed, interactive videodisc instruction was more 
effective than conventional approaches to instruction. 
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2. The more the interactive features of interactive videodisc technology are used, the more 
effective the resulting instruction. 



3. More positive results were obtained for interactive videodisc applications in higher 
educaticm settings than in military and industrial training. Training - as opposed to 
educational - management practices noost probably accounted for these differences by 
training directly to specinc tasks and limiting achievement to thresholds of performance and 
knowledge required by the tasks. 

4. Directed^ tutorial approaches are more dfective than fneeplay simulations in interactive 
videodisc instmction. 

5. Interactive videodisc instruction is equally effective for imparting knowledge and 
improving task performance skills. 

Cost^Effectiveness! 

Interactive videodisc instruction is a viable cost-effective alternative in training and should 
routinely be considered as an option in the design of military training programs. 

TmieonTask : 

Interactive videodisc instruction can increase the efficiency with which training time is 
used by increasing opportimities foe practice and requiring students to participate more 
actively in instruction. One study was found that addressed this issue, and its results 
supported this possibility. 

Rctertion : 

Given the nature of interactive videodisc instruction, it is reasonable to expect it to produce 
equal or longer retention of skills and knowledge. Three studies were found that addressed 
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this issue, and their results suggest that retention under interactive videodisc instruction is 
at least equal to that found under other instructional approaches. 

Final Comment 

The results of this study indicate that interactive videodisc instrucdon is a promising 
iqjproach, that it can have a substantial impact on the return from resources allocated to 
military training and education, and that if vi<;orously pursued it will have a significant 
impact on people-related aspects of military systems and consequendy on our security. 
While there are research questions yet to be answered concerning tids technology, it is clear 
that it has matured sufficientiy to be enq}loyed in many areas of military training and 
education. 
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L INTRODUCTTQN 



Purpoy 

Tm study was pcrfonncd in rcspcmsc to the 1989 I>partmcnt of Defense Appropriations 
Bill directed the Department of Defense to ccxiduct a study on the use of interactive 
videodisc technology in training and education as it pertains to effectiveness, cost- 
efifectiveness* time on task, retention, and overall applicability to cunent and future DoD 
training and education requirements. 

The Office of the Secretary of Defense (OSD) has an empirical investigation already in 
progress concerning the effectiveness of interactive videodisc instruction. This 
investigation addresses issues raised by the above direction fiom Congress, and its 
Executive Summary is included with tiiis report as Appendix A. On die other hand, the 
Departnaent of Defense has been a leader in the develq)ment of interactive videodisc 
technology and has coo^leted many evaluative investigations of its applications in 
instruction. For this reason, a quantitatively-oriented, analytic review - a "meta-analysis" - 
- of what has been learned about the effectiveness and cost-effectiveness of interactive 
videodisc i^lications in military training, industrial training, and higher education was 
undertaken and completed in response to the Congressional direction. Although several 
such reviews have been completed for other technology-based inscructional approaches 
such as computer based instmction, this review is the first of its kind undertaken for 
interactive videodisc instruction. 

Background 

The challenges faced by military trainers are substantial now and can be expected to 
increase in the future. There are at least five reasons for these challenges. First, the supply 
of people available foe military service is decreasing. The number of people reaching age 
18 each year has been dropping substantially. This decrease is not expected to level off 
until the mid-1990s. 

Second, the number of military systems is increasing. In keeping with modernization and 
battlefieki demands for technology, the number of materiel systems fiekled by the the U.S. 
military has increased substantially since Wwld War H. If no technological changes were 
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made in the complexity of military systems, their quantity and variety would by themselves 
increase the challenges to military training. 

Thirds the technological complexity of military systems is increasing. Whether or not the 
complexity of military systems translates into increased job complexity is arguable. 
Nonetheless, the demand for people prepared to hold jobs classified as t(x:hnical or highly 
technical has increased across the Services. 

Fourth, costs to conduct training have risen in both absolute and relative terms. Fuel and 
ammunition have been major contributors to military training costs. Fuel costs will never 
letum to pre- 1970 levels, and ammunition costs continue to rise along with the 
sophistication of our new systems. 

Fifth, Reserve component training poses particularly difficult requirements. The role of the 
Reserve Forces in the Total Force cleariy shows that the Reserv^es are no longer in reserve. 
They wiM be anKxig the first forces conmiitted to battle in any future war. However, the 
Reserve components have a limited anx>unt of time to train, units are widely dispersed 
thtou^out the country, units are not fully equipped, and only a small full-time force of 
qualified supervisors and trainers exists in Reserve units. 

It is not surprising to find military trainers seeking new approaches to mee^ these 
challenges. It is also rot surprising to find them turning to our most powerful new 
technologies for these approaches. Interactive videodisc technology has been prominent 
among these. Like computer based instruction, interactive videodisc instruction r:xiuires 
students to participate actively in the training environment, it allows students access to high 
quality simulation of devices that could not otherwise be made available to them, and it 
distributes both the content and interactions of high quality training to widely dispersed 
sites. As a consequence, the Department of Defense has been a leader in developing new 
instructional approaches based rn interactive videodisc technology. 

This approach involves coupling videodiscs and videodisc players with computers. 
Generally, the videodisc serves as a stOTage medium foe much of the curriculum data base 
and the computer controls how and in what order the curriculum material is presented to 
students. In this sense, interactive videodisc instruction is a form of computer based 
instruction, but the substantial power and numerous capabilities added by videodisc 
technolof^ serve to place it in a catcgoiy by itself. 



The videodisc itself u;^ a reflective, metalized plastic disc on which inforaiation is coded 
as tiny pits (about 0.5 microns in diameter) which are pressed into a transparent s Jostrate 
with a reflective coating. During play of the disc, the pits modulate a laser signal tiiat is 
decoded into audio and video signals. About 15 companies possess manufacturing 
methods and processes to master and replicate such discs from a master videotape 
submitted by a curriculum developer. The firs* copy of a disc may cost $2,(X)0-$3,000, 
but additional copies cost less diar. $20 depending on quantity. The Department of Defense 
has specified die Laser V.sion formal for production, and its approaches to interactive 
videodisc instruction use videodiscs as very large read-only storage for some or all of die 
curriculum data base. 

Videodiscs are available in several sizes - tfie most common are 8 inches and 12 inches in 
diameter. The 12 inch interactive videodisc can store 30 minutes of fiiU-color video, or 
54,000 video frames, on each disc side along widi tv^'o tracks of audio. With audio 
compression, 150 hours of audio can be stored on each disc side. Videodiscs can store 
digital as weU as analog data. Depending on size and fora» t they can store 200 3000 
naegabytes (assunnng 8-bit bytes) of information. An interactive videodisc system is an 
audiovisual instructional system using a "personal" microcomputer, videodisc player, 
cionitor, and special interfaces for displaying graphics and controlling tiie player. Such a 
system will generaUy cost $4,500-$8,000. The videodisc player and its interface typically 
adds about $2,500 to a personal computer system. 

New optical technologies such as CD-ROM (compact disc-iead only memory), CD-I 
(compact disc-interactive), and computer generated imagery wiU compete for die market 
cuirenUy held by interactive videodisc systems based on 12 inch discs. However, die 
instructional capabilities - die functionalities ~ of interactive videodisc systems are die 
essence of a study such as tiiis one. Whatever die hardware media eventuallv chosen to 
deliver diem, die functionalities currentiy made available by interactive videodisc 
technology well deserve assessment for tfieir cunent and potential impact on instruction. 

Much of die development of instractional functionalities now found in botfi con5)uter based 
instraction and interactive videodisc technology has been supported by die Department of 
Defense (Power On!, 1988). Retcher and Rockway (1986) review die history of these 
contributions and identify five new functionalities developed by die Department of Defense 
for interactive videodisc instraction. These five are die following: 
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1. Sunogate travel. This capability simulates a path of travel selected by the user over an 
aita stored on videodisc. Areas chosen fen* initial denoonstration of this capability include a 
town (Aspen, Colorado), a harbor, a nuclear power plant, and an art gallery. Using this 
capabilily, the user may choose the path, control the speed of advance, and direct the angle 
of view while *'tfaveling" through an area using simple controls - usually a joystick* 
When the user conies to a choice point such as an intersection, he/she can turn right, turn 
left, prcxxcd ahead, or go back under joystick control. The videodisc frames the user sees 
originate as filmed views of what cme would actually ree in the area - possibly including 
what one would see at different times of day or at different seasons of the year. 

2. Interactive movies. Qneproblem with training that requires demonstrations of skilled 
perfortiiance is that essential components of the denx>nstrations are invisible to viewers. 
Interactive nx)vies are intended to solve this problem by allowing the user interactive 
control over many aspects of viewing such as direction (front, side, above, below), speed 
(fast, slow, still-frame, reverse), detail (panning and zooming), abstraction (photographs, 
labeled Ime-drawing animations), plot (different actions at different choice points), and 
simultaneous action (gauge readings, actions outside of the cuirent angle of view). 

3. Microtravel. This capability combines sunogate travel with interactive movies and 
allows travel throug^ places where people cannot go. One demonstration of this capability 
involved travel through a jeep engine while it was running. 

4. Spatial data management Electronic libraries in the form of spatial data management 
systems allow users to "fly-over" spatially organized data using joystick controls. Data 
elements are associated with familiar terrain (e.g. a university campus) so that anyone 
familiar with the terrain can locate data of interest - informaticm on chemistry would be 
found by "flying" to the chenustry department, information on tuba concertos might be 
found by "flying" to the tuba section of the university symphony orchestra found in the 
music department, and so on. 

5. Low cost portable simulators. One of the first military skill training videodiscs was an 
Army tank gunnery trainer. This interactive videodisc simulation uses motion video 
segments stored on videodisc to provide a tactical environment and targets. The student 
selects anmiunition, aims, and fires using computer generated imagery overlaid onto the 
video display to receive round trajectory and burst on target feedback. 
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Of the I4)pn>xiinatelj 100,000 videodisc systems currently in non-coni;umer use in the 
United States, about half ait used for training (Miller, 1987). The effectiveness of thtse 
qrplicadons is of significant and natural iiicen^t both inside and outside the Department of 
Defense. The effectiveness and cost effectiveness of these applicaticms are the primary 
topics of the study repeated here. 



n. APPROACH 



Hiis study benefited fixxn other reviews of interactive videodisc instruction. The^c ic^Hews 
include those by Bosco (1986), DeBloois, Maid, and Hall (1984), DeBloois (1988), and 
an annotated bibliognqphy compiled by Sarii, Nau, Martin, Kanitz, and Williams (1988). 
On die basis of these reviews, it was evident that die state of our knowledge would support 
a quantitatively oriented, analytic review of interactive videodisc effectiveness. This review 
forais the con of die study documented here. 

Technical Approach 

The methodology used for analytic reviews has changed somewhat in die last 10 years. 
The lyox score** approach which eariier characterized die methodology has been replaced 
by *'meta-analysis»** In die box-score approach, studies in which an experimental group 
exposed to die treatment under review are collected, die proporticxi of studies in which the 
experimental groiq) means exceed control gmup means by some statistically significant 
extent are calculated, and the treatment is reported as favorable or not depending on 
whedier dus propcvticxi is large or small. Hedges and OUdn ( 1980) have shown diat die 
box score approach has very low power for die treatment effect sizes and sample sizes 
characteristic (£ social science research. They also showed uiat the power of the procedure 
decreases as die number of studies include in die review increases. 

Glass (1976) proposed an alternative approach. Since he was performing an analysis of 
analyses, he described his alternative as '*meta-anal>tic." It differs from die box-score 
s4>proach in three ways: (1) studies relevant to die issue at hand are collected using clearly 
defined procedures diat can be replicated; (2) a quantitative measure, "effect size," is used 
to tabulate the outcon!^ of all die collected studies including those with results that are not 
statistically significant; (3) statistical procedures are used to synthesize the quantitative 
measures and describe die findings of die analysis. Glass's approach appears to be 
especially appropriate for syndiesizing die results of instructional research, and it has been 
widely used for tlil« purpose since its introduction. 

Meta-analysis is a new technique and some matters concerning its use remain unsettled. 
The issues receiving the most attention at present appear to concern die "file-drawer 
problem" and calculation of effect size. Additionally, the use of meta-analysis to perform 
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research syntheses that include costs, i.e., syntheses involving cost-effectiveness, remains 
unaddressed and undetermined. 

The file-drawer problem stems from the reluctance of professional journal editors to include 
studies whose results fail to reach statistical significance - these studies, therefore, remain 
in file-drawers. The questi<»i, then, is how much effect would these inaccessible studies 
have on die results €i our meta-analyses? After some analysis, the answer according to 
RosenAal (1984) and odiers seems to be not much. In this study, it will be noted that steps 
were taken, perhaps inadvertcndy, to alleviate the file-drawer problenL 

Effect size is usually defined as the difference between the means of two groups divided by 
the standard deviation of the control group. Glass, McGaw, and Smith (1981) suggest that 
choice of die denominator is critical and that choices other dian the control group standard 
deviation are defensible. However, they endorse the standard choice - using die control 
ffovip standard deviation - and diat is the approach used here. 

A quantitatively oriented syndiesis of costs and meta-analytic measures of dfect has yet to 
be devised. An approach using statistically adjusted conqiarisons based on analysis of 
covariance is currendy being developed by die Instimte for Defense Analyses, but it is as 
yet too experimental for use in studies such as this one. Moreover, the anmunt of 
systematically collected data now available on die costs of interactive videodisc 
development and use is presendy too small to suppcnt such an analysis. 

Dflfa rfil]ecrinn 

This Study was intended to pursue a meta-analytic approach as closely as possible. 
Defense Technical Information Center, Educational Resources Information Center, and 
Psychological Abstracts data bases were searched using all combinations of die following: 

[Gmiputcr <or> videodisc] 
<and> 

[Assisted <oi> Aided <or> Mediated <or> Managed <or> Based <or> <Empty>] 

<and> 

[Education <or> Learning <or> Training] 




Additionally, we used the following terms by themselves: 

Interactive Videodisc 
Interactive Video 
Interactive Courseware. 

This search piocess turned up more dian 2400 candidate studies. Most of these studies, 
however, concerned ccnqiuter based Im^ouction alone and not interactive videodisc 
instruction. It was nor without value, liowever, and about 10 of the 23 studies listed in 
Table 1 were identified tiirough diis search. On the other hand, there were studies that 
were known to exist but did not appear in this search. Some of these studies are 
unpublished and not catalogued anywhere. Others are published but evidendy not 
catalogued using any of the above keywords. Still others exist only as laboratory reports. 
Whatever the case, we were fofced to decide whether or not to include stu^iies that did not 
turn up in our formal, meta-analytic search. We chose to include them, and we decided to 
search for mcne. 

To increase the basis of evaluation studies, we contacted cognizant individuals in the 
training commands of all three Services (US Army Training and Doctrine Command, US 
Navy Chief of Education and Training, and US Air Force Air Training Command), in the 
personnel research and development commands of all three Services (US Army Research 
Institute, US Navy Personnel Research and Development Center, and US Air Force 
Human Resources Laboratory - smong others), and in specific Service schools where we 
knew interactive videodisc materials had been used 

We also c(Mitacted industrial training organizations to see if they had evaluation data and if 
they were willing to release it to us. We pursued this by publishing a notice in the 
Videodisc Monitor, by calling eveiy developer listed in the IBM's March 1988 
InfoWindo w Courseware Catalog, and by calling every developer listed in the 1988* 1989 
Videodisc Compendium far Education and Training. With these actions we hoped to 
unearth "fugitive" documents that had reported studies performed for business but had not 
been published Few such documents emerged. We were impressed by the willingness of 
people to root Uirough their personal files for us and dig out references that were already in 
the published literature. Some of these references had not been identified by our search of 
the data bases, and we were glad to know about them. 
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Finally, wc called a number of researchers in universities. As usual we encountered 
hdpfiil facilitating people who were willing to dig through their personal files and send us 
whatever they felt would be relevant 

In neariy every case, our telephone calls led us to additional individuals. We are grateful to 
allwhohelped We contacted over 250 organizations, and they are listed in Appendix B. 

The criteria we zpphcd to the studies that turned up were systematic and likely to better 
satisfy meta-analytic purists. All tfie usual criteria for care and control in the conduct of 
evaluation research smdies were applied These are the criteria diat are likely to be 
discussed in any good university course on the design and conduct of experiments in the 
social and behavioral sciences. Additionally we applied the following three criteria: 

L We required studies that involved a comparison an interactive videodisc approach 
with some other approach to instruction* It is possible, if not desirable, to evaluate an 
instructional iq)proach witiiout a comparison. For instance tracing a "trajectory" of student 
progress tiirough a course as described and applied by Suppes, Fletcher, and Zanotti 
(1976) is more powerful than the experimental group, ccmtrol group compariscHis we 
sought here. However, we needed comparisons to calculate effect size, and we excluded 
studies without them. 

2. We required studies that reported sufficient data to peraiit calculation of effect size. 
Generally, we needed means and standanl deviations for all treatment groups. In tfie 
absence of tiiese data we were often able to estimate effect size from otfier infomiation 
provided - especially information obtained fixmi significance tests. Glass et al. (1981) 
describe a number of ways to calculate effect size when means and standaid deviations are 
not provided directly We used tiiesc when we could, and we included all tfie studies tfiat 
we could 

3. We required smdies tiiat involved students who resemble the ntilitary personnel who arc 
our primary ccxicem. We did not include studies involving children or handicapped 
individuals* We found no candidate studies involving non-handicapped children, but we 
did not search as assiduously for tfiese as we did for otfiers. We did encounter 2-3 studies 
involving handicapped students that were otherwise acceptable, but we did not include 
them in this study. 
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Tabularion of Findings 



Our seaich identified the 3 1 studies itfeitnced in Table L The taUe is organized so that 
militaiy training studies are listed first, industrial training studies are listed next, and higher 
educaticm studies are listed last Within these groups, the studies are listed chronologically. 
We did not exclude secondaiy school studies, which would have made up a fourth 
instructional setting category; oone were identified in our search. Two studies were 
performed at the US Air Force Academy (Crotty, 1984; Verano, 1987). Although these 
studies used military subjects, the setting seenaed nxxe characteristic of higher education 
dian military training, and they are iiKluded with the higher education studies. 

The main distinction in the instructional approach column is anxxig those applications that 
used the videodisc as a tutorial, programmed textbodc - the intrinsic programming 
approach advocated by Crowdcr (1959) - those that used the videodisc to permit free 
exploration of simulated situations or equipment, and those that used both by combining 
tutorials with simuJaticms. 

Disc levels were assigned in keeping with the definiticxis developed by ihe Joint Service Ad 
Hoc Committee for Interactive Courseware I>ata Item Descriptions. This committee based 
its definitions cm those develq)ed by the University of Nebraska Videodisc Design and 
Production Group. These definitions in turn were derived finom common developer usage. 
They are the following: 

Level 0 - A videodisc system intended for linear play witliout interruptim. 

Level I - A videodisc system with still/freeze fiiime, picnire stop, frame and chapter 
search, dual channel audio, but no programmable roenuny. All functions are intended to be 
initiated by manual inputs from the videodisc player's keypad. Picture stop and chapter 
stop are read from the videodisc. 

Level n - A videodisc system with on-boaid, programmable memory. The videodisc 
player's memory is programmed by "digital dumps" from audio channel two of the 
vkieodisc or by manual entry from the videodisc player's keypad. Inputs are made from 
the keypad or fixnn a device that emulates the keypad. 



10 



Level m — A videodisc system in which the videodisc player is interfaced to an external 
computer. The videodisc player acts as a computer peripheral with its functions under the 
computer's control. 



Level IV - A videodisc system in which the videodisc player is interfaced to an external 
cooqxiter and the videodisc is used to store digital data intended to be read and used by the 
computer. The videodisc functicxis both as an optical storage device for digital infonnation 
ami as the source of analog picture and sound 

All five definitions are listed here for the leconL Only two levels (U and ID) were found 
among the studies listed in Table 1 . The term 'interactive videodisc' in ccmtrast with 
Videodisc' alone typically inq)lies the presence of a conq)uter, and it is used to refer to a 
Level m or Level IV videodisc. 

Outside of what could be determined by descriptions of the instructional approach and the 
level of videodisc used - and of course what can be determined fiom the evaluation results, 
no direa review of interactive videodisc content was attenqited in this study. We did not 
attempt to review such features as the quality of graphics, clarity of instructional text, and 
verisimilitude of the simulations stored on the videodiscs used in the evaluation studies we 
found. 

The 'N* column lists the number of subjects in the e;q)erimental (Exp) and the control (Ctrl) 
groups, and the Comparison column tells what the experimental and ccmtrol treatments 
were. Generally the experimental group involved interactive videodisc (IVD) 
presentations, and the control group did not In some cases, different interactive videodisc 
presentations are conq)ared with each other, and in one case a comparison of computer 
based instruction (CBI) with the control treatment was included in the smdy and is reported 
in the table. A common treatment in maintenance and operattn* training involves the use of 
interactive videodisc to simulate actual equipment (AE). 

The basic distinction in the Outeome colunm appears to be between knowledge and skill, 
that is to say between those studies that assess students' knowledge of the presented 
subject matter and those studies that assess students' skill in performing some criterion 
task(s). 
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Efift^t size was calculated as the difference in the group means divided by the control group 
standard deviation* In some cases, effect size was estimated using procedures suggested 
by Glass et aL (198 1). Also in some cases, the means and standard deviations were not 
lepocted in the referenced study, but they were obtained through personal communicadon 
with the study author(s). 

Some studies involved simultaneous conq)arison of more than two treatment groups and 
began widi an analysis of variance (ANOVA) before proceeding to compare the treatment 
groups in a pairwise manner. The statistical significance of the analysis of variance 
comparison is reported in the table. All effect sizes reported were calculated from pairs of 
treatments. Generally, effect size was calculated so that it was positive if the difference 
favored die videodisc treatment and negative if it favored die control treatment More 
specifically, and to cover cases such as those in which different videodisc treatments were 
compared with each otiier, effect size was always calculated so that it was positive if it 
favored die instructional treatment judged to be the toosi interactive of the pair. The 
treatment judged to be naost interactive was always the first of the two treatments listed in 
the conesponding Comparison column. 

The abbreviation *ns* is used in die Effect Siit colunm to indicate that the observed 
difference in treatments was not statistically significant Odierwise, die significance level 
reported in die study is listed The abbreviation "na* is used in die Effect Size and die Cost 
Ratio column to signify that the data of interest were not available and not reported 

The signs of values reported in the Percent Improvement column were determined in a 
manner similar to that used for the Effect Size column. It was calculated so diat it is 
positive when the difference favored the most interactive treatment, i.e., the treatment listed 
first in the corresponding Comparison column. 

Cost ratios are reported as the ratio of videodisc treatment costs over the control treatment 
costs, or, again, the most interactive treatment costs over the other treatment costs. The 
closer the proportion is to zero, die more fiivorable it is to the mwe interactive (generally 
the videodisc) treatment Generally, we did not find costs to be reported in diese studies, 
aldiough many audiors acknowledged cost and cost-effectiveness to be significant issues. 
Cost models usually cover costs in four categories: research and development, initial 
investment, operating and support, and recovery and salvage. All cost ratios found and 
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reported in Table 1 are based on initial investment (H), operating and support (OS) costs, 
or both. 
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m. FINDINGS 



Effecriveness 

Indications of the effectiveness of interactive videodisc iqjplications in training and 
education can be obtained by combining the <^ifectsi2»srepo^ Different 
combinations will yield differrnt results, but it will be noted that the overall picture is 
favorable. 

Not all die effect sizes reported in Table 1 are appn^ate for this study. We need to select 
more carefully the effect sizes we actually use to calculate an overall average. In making 
tfiis selection we eliminated three sorts of findings from the calculation: 

1. Some of the effect sizes reported in Table 1 are included for c(xnpleteness and involve 
coaq;>arisons against control treatments in which no relevant instruction was provided. 
These coopuisons were included in the original studies to detenuine if interactive 
videodiscs teach anytiiing. As might be expected, the effect sizes reported for these 
coinpariscMis are large and favOTaUe. Of die seven effect sizes reported for these 
comparisons, four are in excess of 1 .00. However, these comparisons do not involve two 
different approaches to providing instruction, and they were eliminated from further 
calculations. 

2. Other effect sizes reported in Table 1, and also included for completeness, do not 
involve interactive videodisc applications at all. Three of die post- ANOV A pairwise 
comparisons reported by Levenson et al. and one repwted by Gibbons, et al. did not 
involve interactive videodiscs, and their effect sizes were eliminated from further 
calculations. 

3. Finally, studies involving Level II videodiscs were also eliminated from the calculation. 
This is in keeping with the usual definition of interactive videodiscs. 

This selection does not exclude effect sizes based on comparisons of different levels - or 
"intensities" - of interactive videodisc treatments such as those reported by Cicchinelli et 
al. (1984) and by Vcrano (1987). In diese ccmparisons, die treatment that most used the 
interactive capabilities of die technology - the most intensive treatment - was considered 
as the experimental approach and die odier as the control. 
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As a result of these procedures, we included 95 effect sizes from Table 1. Of these, 78 are 
positive and indicate positive results favoring the most interactive treatment of the treatment 
group pairs considered The effect sizes range from 4.14 to 2.49. Their mean is .51, and 
their median value is .46. This average is 20-90 percent larger than effect sizes reported in 
meta-analyses of computer based instruction (cf , Niemiec and Walberg, 1987). 

Although effect size appears to be an excellent metric for aggregating the results of many 
different studies, its in[q)licatioas for the practical worid of instruction are not inmiediately 
evident Roughly, a me ui effect size of .5 1 from a sample as large as 95 suggests that, 
through tiie use of interactive videodisc technology, a trainer can expect to increase the 
perfomiance of his/her graduates by about one-half a standard deviation above their present 
level of achievement . This is, roughly, to say that his^er average graduate, the graduate at 
the 50th percentile, could be perforaiing at the level of his graduates now at the 70th 
percentile tiuough die use of interactive videodisc technology. 

TaUe 1 was intended to invite exploration of die results. We calculated the averages for 
military, industrial, and higher edxatim settings. The average effect size for military 
applications, based on 44 observations, was .35 - suggesting a rise of 50th percentile 
students to the S4th percentile. There were seven observations for industrial settings. 
Their average effect size was .30 • suggesting a rise from the 50th to the 62nd percentile. 
The average effect size for instructional applications in higher education was based on 44 
observations. It was found to be .70 - suggesting a rise from the 50th percentile to the 
76th percentile. 

Another exploration oxicemed the instructional t^proach and the type of instructional 
outcome assessed. We categorized the studies into three groiq>s based on their instructional 
approach. These groups keyed on treatments in which the instructional approach was 
directive and tutorial, those in which the instructional approach was based on freely 
explored simulations, and those that combined both tutorial and simulation approaches. 
We also contrasted studies in which the instructional outcome assessed was course 
knowledge with those in which the instructional outcome assessed was skill or 
performance. The results of this exploration are shown in Table 2, which also shows the 
average effect sizes fm all mtorial approaches, all simulation approaches, all combined 
approaches, all knowledge outcomes, and all performance outcomes. 
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The results shown in Table 2 indicate that directive* tutorial {^proaches yield greater 
instructional results Aan simulation {y)pn>aches. Of course* even if there were no 
differences in instructional effect between interactive videodisc simulations and use of 
actual equipment for instruction, tfie considerable cost avoidances to be realized from using 
videodisc simulations rather Aan actual equipment - as well as the subf^tantial increase in 
(^q)ortunities for laboratory-type practice — would continue to argue strongly for the use of 
interactive videodisc i^lications in instruction. 

Secondly, the results shown in Table 2 suggest an advantage for instructional outcomes 
based on knowledge compared to those based on skill performance. Here, confidence that 
the instractional ou>x:ome was correcdy categorized should be high, but the difference 
between knowledge and performance outcomes could, for instance depend on the fact that 
about S3 percent of the performance results came from studies using the more successful 
tutorial s^proach either by itself or in combination with a simulation whereas 77 percent of 
the knowledge results came from studies using a tutorial ^proach. The difference between 
knowledge and performance outcomes may be explained by these differences. 

Finally, it should be noted that even die smallest of these effects, undirected simulation to 
achieve performance outcomes, shows a positive average effect size of .05, which indicates 
diat interactive videodisc instruction used in this way is at least as effective as more 
conventional alternatives. A littie more should, and will, be said about the cost avoidances 
that can accrae from the substitution of videodisc simulated equipment for actual equipment 
in instruction. Even negative instructional outcomes requiring more time to reach criterion 
can prove to be cost-effective because of the relatively small cost of simulated equipment 
tfiat reacts witii the kinds cr realism needed to achieve instructional criteria, 

Siudent time savings of about 30 percent have been no^ in analyses of computer based 
instruction starting witii die widely noted Orlansky and String (1977) study and continuing 
on into the present Only four studies in Table 1 report savings in the time students needed 
to reach criteria. The average dfect size foe time saved across these studies was .93. 
Interestingly, the average percent time saved across tiiese four studies was 30 percent. 
Four studies do not provide a conclusive base, but tiiis result suggests tiiat the time savings 
for interactive videodisc instruction may be about the same as those for computer based 
instruction. 
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All Ac above results arc promising, but they arise directly and in a straight-forward manner 
from available evaluation data on the effects of interactive videodisc applications in training 
and education. There are some final matters to note concerning the studies fiom which 
these results were drawn. 

1. Most of the evaluations reported here were perfcxmed by developers who produced the 
interactive videodifc instruction being evaluated rather than by third pa^ There are 
strengths and weaknesses in this app;x)ach. Developers are rarely indifferent to the success 
of their products and might, intentionally or unintentionally, bias the results of their 
evaluation. On the other hand, there m standards for the performance and documentation 
of evaluation studies. Observance of these standards can be assessed by others — as they 
were in die search for evaluation studies performed here - and the pc.^ibilities for bias in 
studies performed by well-intentioned developers should be minimal. Also, developers 
have a stake in an honest assessment of tiieir products. Most of the developers and 
evaluators referenced here cje scientists and technologists, not marketeers. At base, most 
evaluations are foraoative - intended to suggest inqirovements and modification in the 
product and in the technology - not summative or intended to prove its value in some 
conclusive manner. The better the information developers get, the better their products can 
become. Hnally, developers are in the best position to evaluate their products. They 
understand better than anyone - including the potential users of their products - the 
probable strengths and limitations of vAiat they have developed, and they are better 
prepared than anyone to devise an assessment of their product 

2. One real weakness of any evaluation of any new technology is that there is simply 
nothing else like it Each new technology has its own strengths and weaknesses. If the 
evaluation is held to strict instructional controls based on one technology, the other, 
alternative technology will be at a disadvantage. This problem can be ameliorated 
somewhat, but not entirely, by focusing on the instructional outcomes and not on 
specifically what is done to accomplish these outeomes. Even in the best of situations, 
however, new approaches are unlikely to be used to best advantage. Among ail the 
evaluations listed, we probably have not yet seen an optimal use of an instructional 
approach based on interactive videodisc technology. 

3. A related problem arises in the comparison of a newly introduced instructional approach 
with an existing one. Very often, as Orlansky and String (1977) found, the instructional 
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m a t erials prepared for the new approach are trimmed down and focused on the instructionai 
outcomes, but the existing approach is, naturally enough, left as it is. In this way, some 
evaluations may be unintentionally biased in favor of the new approach. 

In short, it is as easy to argue that die view of interactive videodisc applications seen here is 
overly optimistic as it is to argue diat it is overly pessimistic. At present it seems best to 
rely on the data we have and to conclude that applications of interactive videodisc 
technology to the problems and processes of instruction offer genuine and demonstrable 
instructional gains over the approaches we use now. 

Cost-Effectiveness 

There are nine cost ratios reported in the studies listed in Table 1. Two of these are for 
initial investment costs (II), tiiree for operating and support costs (OS), and four for initial 
investment costs and q)erating and suppmt costs combined These ratios are all calculated 
as die costs of interactive videodisc instruction over the costs of conventional instruction 
generally involving use of equipment The ratios cited for initial investment costs are .27 
and 4.49; tiiose for operating and support costs are.03, .61, and .02; those for initial 
investment and opcratmg and support costs combined are .94, .34, .44, and .05. 

There are other studies to be mentioned in this regard. All these studies explicidy assume 
that the interactive videodisc training is at least as effective as the conventional training with 
which die cost comparisons are made. Doughty and Lent (1984) present two such 
comparisons, one of which involved actual costs. This comparison concerns a training 
facility for a large jet engine remanufacturing facility. Doughty and Lent estimated the total 
costs for procuring, installing, and maintaining interactive videodisc instruction foe this 
facility over a ten year period to be $4,419,000. Three, sources of migor cost avoidances to 
be realized £rcxn installation of interactive videodisc instmction were identified. These are: 
engine failure avoidance, training cost avoidance, and staff savings. Doughty and Lent 
estimated die total ten year savings firom diese tiiree sources alone to be $14,031,000, 
yielding a return on investment of 316 percent 

Many commentators have remarked on the advantages of computer based instruction and 
interactive videodisc instruction to deliver standardized, decentralized training to students — 
rather dian require tiiat the students be delivered to die training. Walker (1985) presents an 
industrial training study in which the costs of delivering interactive videodisc instruction to 
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remote sites is con^aied to the costs of centralized training. When the initial investment 
costs for developing and installing the interactive videodisc training ait amortized over 
Arte years Walker showed that the costs per student are $1^568 for the centralized training 
and $SS3 for the intenu:tive videodisc training - a cost ratio of .35 for coubined initial 
investment and operating and support costs. 

After an extensive study Dn the feasibility, costs, and effectiveness of various methods for 
training smog check mechanics, Maher (1988) concluded that because of the eliminadon of 
instiuctois and reduced tndning time, vidi odisc instruction wouki pixivide the nx>st cost 
effective qiproach of five that were considered for han Is-on mechanic training aiu: 
verification testing. Maher found that the costs for videodisc training would be $50.60 per 
student coii4)ared with baseline costs of $ 102.78 per student - a cost ratio of .49 for 
combined initial investment and operating and support costs. 

The substitution of interactive videodisc simulated equipment for the real thing - the actual 
equipment - will likely prove a significant source of cost avoidances. This issue was 
rarely addressed direcdy - widi real costs reported - in die studies reviewed here. It 
shoukl be noted tiiat a sinular question addressed by Fletcher and 
substitution of conq)uter based simulations for actual equipment in conq)uter based 
instruction found cost ratios oi simulated equiprient to actual equipment diat were as low as 
.01. Basically, these cost i ^tios can be as low as we want, depending on the actual 
equq)ment being simulated 

Another importai. . area of cost avoidances is student time. If the 30 percent savings 
reported for students to reach threshokl levels of performance obtains for interactive 
vkleodisc instruction as it does for conq)uter based instruction - and it seems reasonable to 
expert tiiat it will Uien savings for some high student load courses may reach millions of 
dollars. A possibly more inq)ortant source of savings suggested by Solomon (1986) 
woukl result from die reduced numbers of people the military would have to support in a 
job category by reducing the time needed to train people fm it 

Qearly, the cost-effectiveness of interactive videodisc training can vary widely. It will 
depend on the objectives of the mstniction and the resources required for other approaches, 
whether tiiey are conventional or not. It appears to be an issue that cannot be determined in 
general, but must be deterimned on a case-by-case basis. Gn the other hand, there are at 
least some instances in which interactive videodisc instruction is the preferred cost-effective 




alternative, and as it becomes more conuiKxily considered rl:^ aiay turn out to be many 
instancf jx wii it is the preferred alternative. 

Time on Task 

Very little direct evidence exists to suggest that students using interactive videodiscs are or 
are not willing to spead more time in instructicxi or diat tiie time iiey do spend in 
instruction is or is not more task centered dian is the case for more ccxiventional fcvms of 
instruction. A number of studies listed in Table 1 surveyed students* opinions of die 
interactive videodisc instruction they received. By and large these studies report that the 
students enjoyed the interactive videodisc instruction, they woukl recommend it to others, 
and they found it easy to use, all of which suggests diat they might be willing to attend 
more directiy to instmction presented using interactive videodiscs than they would to 
instnictkm presented using niore conventional approaches. Student ratings of interactive 
videodisc instruction mi^t well be attributed to a novelty effect and be expected to wear 
olff. However, Baggett(1989, Personal Conomunicaticxi) reports that in her experiments, 
some of which are quife lengthy, it has not Over time and extended use students continue 
to report high interest and enjoyment of interactive videodisc instruction. 

Winkler and Polich (in press) investigated interactive videodisc instruction used to 
supplenoent actual equiimient experience for military radi"^ operators. They found that 
introduction cf die interactive videodisc materials led to a4S percent increase in the time 
students spent practicing radio installation. In effect, these students received increased 
pn«.dce without increasing die amount of time they spent in the training course. In other 
words, they increased the amount of time they spent on the learning tasks. 

Retention 

Given the involving, if not compelling, nature of much interactive videodisc instruction, it 
seems reasonable to expect it to be memorable - skills and knowledge obtained under 
interactive videodisc instruction shouM be retained at least as IcMig and pertiaps longer than 
skills and knowledge obtained under other instructional approaches. Thite of the studies in 
Table 1 - Bundersoi et al. (1984), Verano (1987), and Young and Tosti (i 981) - 
addressed retention. All were concerned with the retention of knowledge rather than skills. 
Effect size for the two measures reported by Bunderson et al. reduced from an average of 
.60 to .41 over die 1-week retention inteival studied. Effect size for the three measures 
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itpcmed by Verano itduced from an average of 1.86 to 1.28 over the 4- week retention 
interval studied Effect size for the single measure reported by Young and Tosti m 
ficm -•39 to +,14 over a retenticxi interval of 10 weeks. 

In tliese three studies, the experimental groups and the control groups seem to have grown 
together over die retention interval. How quickly their differences decrease and whedier or 
not all dififerences eventually disappear are questions that cannot now be aiiswered using 
available data and that await further research. 

Some recent research suggests that retention is very sensitive to the choice and sequence of 
instructional media. Baggett (1983) showed that performance on assembly tasks assessed 
after a 1-week retention interval was significandy improved for students who had hands-on 
practice first and viewed a film second rather than the other way around Morerecent 
e3q)erimental work by Baggett (1988) has shown no advantage in retention obtained from 
hands-on practice presented at the same time as audiovisual instruction - groups widiout 
practice performed as well as groups with practice under these conditions. She did show 
an advantage in both immediate and 1-week delayed perfomuuK^e for practice a^ 
audiovisual instruction tiiat was presented sequentially and not simultaneously - with the 
best results again being shown by groups who received the hands-on practice first followed 
by the audiovisual instraction and not the other way around This result is notably in 
keeping widi the informal observations of experienced military trainers diat interactive 
videodisc instraction appears to achieve better results in advanced training than in 
beginning training applications. 
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IV. rONCLUSTONS 



It should be eniphasized tfiat the results of this study and its conclusions concern 
instructional capabilities - fiincticmalities - not hardware. To refer to these capabilities as 
interactive videodisc instruction is both convenient and intuitively reasonable, but it is the 
functionalities that count in instructicm, not the hardware. 

Hardware systems other than tiiose that couple videodisc players to computers and that 
provide tbt same functionalities as interactive videodisc technology ought to achieve the 
same instructional effects as tiiose observed here. Ai present there is no haxdware 
techndogy tiiat can competitively provide the same instructional functionalities as 
interactive videodisc systems. However, computer generated video, computer generated 
audio, and contact disc technology are all likely to encroach cm tiie territory now held by 
interactive videodiscs. Two in^lications from this observation are: (1) the training and 
education communities need to understand and track new technological develqmients to 
assure tiiat tiie functionalities they need are best obtained from the technologies they use; 
(2) the traiiung and education communities need to understand as well as possible whst 
functicxuQities they genuinely need to accomplish their missions - we need to better 
understand die costs and effectiveness of specific functionalities. 

The following conclusions are suggested by the results reported in this study: 
Effectiveness 

1. Interactive videodiscs can be used to teach. The nine ccxnparisons, reponed in Table 1, 
of interactive videodisc instructiot. witii a "placebo" treatment in which notiiing was taught 
resulted in an average effect size of 2.19 (arise for die SOtii percentile student to something 
in excess of die 98tii percentile) ""or immediate tests of achievement and an average effect 
size of .87 (a rise foe die SOth percentile student to the 80th percentile) for retention 
achievement tests given after some time interval. 

Z Interactive videodisc instruction is relatively mort effective than conventionally used 
approaches to instructicm. Of die 95 comparisons included in the overall statistics reported 
for effect size in tiiis study, 79 compared interactive videodisc instruction to conventional 
iy>prx)aches. These 79 comparisons exclude those that compare interactive videodisc 
instruction with videotape or computer based instruction, or involve comparisons among 
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different videodisc approaches. The aven ge effect size for these comparisons is .9 1 (a rise 
for the SOdi percentile student to the 82nd i>ercentile). 

3. The more the interactive features of interactive videodisc technology arc used« the more 
effective the resulting instruction. There are seven results repented in Table 1 for Level n 
videodiscs. The average effect size for these applications is* 1.53. Six of these results 
caii:e from the Holmgren et al. (1979) study in which Army Training Extension courses 
designed to be presented using audio txpc and fihn slides were directly recorded on 
videodisc. The findings d this study suggest that tape*slide materials presented on a tape- 
slide medium will produce training results that are superior to those obtained when the 
same materials are presented using another medium. This finding should ccxne as no 
surprise. 

These findings further suggest that the less the unique c^abilities of interactive videodiscs 
are used, the less the advantage gained from using the medium. This possibility is 
suppcMted by con^arison of the Wankcl (1984) with the Stevens (1984) results. The same 
basic videodisc material ("Puzzle of the Tacoma Bridge CoUi^se**) was used in both cases, 
and in both cases the same test of physics knowledge was used. When the videodisc was 
used with a Level n q>pn>ach, the effect size was -.17; when die videodisc was used with a 
Level m approach the effect size was +.29. Verano ( 1987) used three levels of 
interactivity in his study (linear, segmented, and interactive). The higher the level of 
interactivity he used, the larger the resulting effect size became. Instructional evaluation 
generally requires a sizable body of research to achieve a conclusive result, but the 
combination of the Holmgren, Verano, Wankel, and Stevens results mate a strong case for 
using the full capabilities of Level m videodisc media in instinctional applicaticms. 

It should be noted that an extrapolation of this conclusion to Level IV videodisc materials is 
not warranted. The step from Level HI to Level IV, under the definitions used by this 
study, does not nnark any inqnovement in the way instructional materials are presented, 
only a change in the way they are stored. For that miatter, die proprietary nature of all 
current sqpproaches to encoding and decoding digital infmnation on and off videodiscs 
coupled with the lack of additional instructional capabilities makes Level IV videodisc 
tec*hnology incompatible v/ith current Defense initiatives to achieve courseware portability. 

4. Interactive videodisc instruction appears to be more effective tiian computer based 
instruction witiiout interactive video. The average effect size of .70 observed here for 
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interactive videodisc instniction used in colleges is considerably higher than both the 
average effect size oi 26 found by Kulik and Kulik (1986) for conoputer based instruction 
in colleges and the average effect size of .42 found by Kulik, Kulik, and Shwalb (1986) 
for ccxDputer based instruction in adult education. The average effect size of .35 found 
here for interactive videodisc q)plications in military training is ahnost cm the midpoint 
between die findings of the two Kulik et al. studies concermng computer based instruction. 
However, tiie effects €i interactive videodisc instruction on achievement in military settings 
have most probably been dan^' by the administrative procedures of military trainers who 
generally seek tfareshokis of achievement, not maximized achievement 

The addition of videodisc ca[ abilities ought to add scxne value to computer based 
instruction since it increases the cost of the hardware system by about $2,5(X) per system. 
Because of these cost-effectiveness trade-offs, and because of the different objectives of 
instructional programs, blanket recommendation for one or the other of these technologies 
appears ill-advised Recommendations concemiiig the jse of computer based instruction 
and interactive videodisc instruction in specific applications will have to be made on a case- 
by-case basis for tiie foreseeable future. However, it should be noted that strong evidence 
exists that each of these q)proaches can be more effective than conventional approaches to 
instruction. 

5. Directed, tutorial approaches are more effective than fireeplay simulati(His in interactive 
videodisc instruction. Evidentiy, guided discovery is worth more than simple discovery in 
instruction. Most of tiie tutorial approaches included in Table 1 included simulaticms of 
situations or equipment, and all the simulations must of necessity have provided feedback 
on tiie success of tiie student's actions. The main difference between the two approaches 
appears to be tiie extent to which the student was guided in his/her instructional interactions 
with die systeuL Guidance appears to pay off The avera^ effect size observed for 
tutorial approaches alone and tutorial approaches combined with simulation was .76 
compared to an average effect size of . 14 for q)proaches that more directiy resemble 
straight simulaticxis. This result is consistent with a body of literature that suggests that 
simulators and training devices are nxne effective when included in a training system than 
when tiiey are simply used as stand-alone resources. 

6. Interactive videodisc applications are relatively more effective in higher education 
settings tiian in nr Jitary training. The average effect size observed in this study for higher 
education was JO, compared with an average effect size of .35 observed for military 
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training iy)plications. This result is most probably due to the nature of military training in 
whidi die objectives are job driven and the practice is to aim for some threshold of 
achievement witfi minimized costs in contrast to instruction in education settings where 
costs are generally set at tfie beginning and the objective is to maximize achievement within 
these fixed costs. This possibility is supported b~ 'he average effect size of .30 found for 
industrial training, which is managed in ways or slosdy resembling military training 
tfian higher education - the atten^t is generall}' to minimize costs to reach specific 
diresholds of achievement ratha than to maximize achievement while holding costs fixed 
Whatever the case, the average effect size of .35 observed for military training is substantial 
- it suggests a rise for tile SOdi percentile studrat to the 64tii percentile. 

7. Interactive videodisc instruction is equally effective tor both knowled^ and skill 
outcomes. The average effect size observed for knowledge outcomes in this study was .59 
(a rise for 50th percentile students to the 72nd percentile) compared with an average effect 
size of .38 (a rise for 50di percentile students to tiie 65tii percentile) observed for 
performance outcomes. These differences are of practical significance, but they appear in 
tfus study to result primarily fixxn tiie greater proportion of simulation approaches in those 
cases concerned witii skill outcomes than from a difference in trying to obtain the different 
sorts of outcomes. 

Cost^Eflfectiveness 

Interactive videodisc instruction is a cost-efifective alternative in training. The instances in 
which it can be demonstrated to be the preferred cost-effective alternative are sufificientiy 
promising to ccmclude tiiat it ought to be generally and routinely considered along with 
other instructional approaches now commonly considered during the design of instructional 
programs. 

Time on Task 

Interactive videodisc instruction may increase time on task. Intuitively, the low cost of 
interactive videodisc instruction relative to the costs of actual equipment should mean that 
students should be able to spend time actively engaged with equipment simulated by 
videodisc technology that they now spend passively watching demonstrations on actual 
equipment Only one study reported an observation relative to this point. Winkler and 
Polich (in press) repeated a 45 percent increase in the time spent practicing radio installation 
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as a result of the introduction of interactive videodisc. This result seems best viewed as 
promising, but not conclusive. M<M:t work is needed which focuses directly on die topic. 

Retention 

Based (xi evidence presented here, there is no reason to believe that interactive videodisc 
instnicdon will have dtha a particulariy beneficial or detrimratal effect oa knowled^ or 
sidll retention. Post-training experiences typically influence retention of knowledge and 
skills in a powerful manner. Defense experience with all new approaches to training has 
been that their promise and payoff resides primarily in improving the efficiency of 
instruction, not in assuring that what is learned will be longer retained. There is no 
evidence to suggest that interactive videodisc instruction will be any different in tiiis regard. 
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V. FINAL mMMRNT 



It is clear firom the results of this study that interactive videodisc instruction is a promising 
approach, that it can have a substantial inq>act on the value cf resources we allocate to 
military training and education, and that if vigorously pursued it will have a significant 
inqMict on the people-related aspects of our military systems and ccxisequendy on our 
security. While there are research questions yet to be answered concerning this 
techndogy, it is clear that it has matured sufificiendy to be widely employed in all aitas of 
military training and education. 

The Assistant Secretary of Defense for Force Management and Personnel has initiated a 
number of acticms to insure that Defense training and education receive the full benefits of 
interactive videodisc technology and conq)uter based instructioiL These actions include die 
development of recoomiended practices for insuring that courseware will be portable across 
a variety of hardware and software systems other than those on which it was originally 
developed, the establishment of data collection, storage, and retrieval practices to insure 
that timely information on die ^iplications of tiiese technologies to Defense training and 
education is readily availaU and the preparatim of a Department of Defense Directive to 
establish policies and procedures for accomplishing these actions. 

The Department of Defense pursues new approaches such as interactive videodisc 
instruction and computer based instmction because it must Business as usual will not 
meet die challenges of fewer peq)le, increased density of military systems, higher training 
costs, and Reserve Component training requirements. As we must modemi; ?ur forces 
by investing in new military systems, so wc must also naodemize our training technologies 
to supply die people we need to operate, maintain, uid employ diese systems at dieir 
intended levels performance. Without the people, L ^se systems will provide us litde in 
increased security. 
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TABLES 



Table 1. Studies of Interactive Videod isc Ttistnicrion Effecri vencss. 

Comparispn Outcome 



Instnictional Instructional 
Settinp Content 



Military 
Training 



Military 
Training 



Instructional Disc 
.teOdCh Level N 



Armor 
Concepts 

Aitillery 
Concepts 

Infantry 
Concepts 

Armor 
Concepts 



Artillery 
Ccxicepts 

Infantr}' 
Concepts 

Electronics 
Maintenance 



Slidc^ape n 
Tutc^ 

n 
n 
n 

n 

« 

n 



Practice on HI 

Simulated 

Equipment 



16Exp 
31Ctrl 

8Exp 
33Ctri 

17Exp 
BlCtrl 

16Exp 
31Ctrl 



23Exp 
ZiCtA 

15Exp 
31Ctrl 

27Exp 
24Ctrl 



rVDvrs 
Sli(k/rape 



IVD with Re- 
view vrs 
Slide/Tape 



IVD only vrs 
AEonly 



Effect Percent Cost 
-Size ImEL Ratio Reference 



Performance 
Te«:t 



-2.69 
p<.C3 

.85 
ns 

-2.70 
p<.05 



Perfomiance -3.08 
Test p<.05 



-.69 
ns 

-2.20 
p<.05 



Paper Test 

ofAE 

Knowledge 

Test Using 
AH 



. 39I.S.K 
ns 

-.3lJ'S.P 
ns 



10-Week .14^'^'^ 
Retention of ns 
Knowledge 



Time to 
Criterion 



-.15JS.P 
ns 



-17% na 
8% 
-18% 
-20% 

-7% 
-15% 
-6% na 

-1% 

2% 

-2% 



Holmgren, 
Dyer, 

Hilligoss, and 
Heller (1979) 



Young and 
Tosti (1981) 

Also 

Kemer (1984) 
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Notes Concerning Effect Size: 

I - Included in Effect Size Overview Statistics; T - Tutorial Approach only; S - Simulation approach only; 
C - Combined Tutorial and Simulation; K - Knowledge Outcome; P - Perfoimance Skill Outcome 
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Table 1 (Continued). Smdirspf l 



Instnictional 
Setting 



Insbtictional 
Content 



Military 
Ihdning 



Electronics 
Maintenance 



Military 
Training 



Electronics 
Maintenance 



Instnictional Disc 
Approach Level N 



Tutorial on 

Simulated 

Equipment 



Tutmal on 

Simulated 

Equipment 



Hft^ Instruction Effectiveness. 

Comparison Outcome 



Effect Percent Cost 
-Size ImpL Ratio Reference 



m 



m 



llExpl CAI(Expnvrs Time to 
20Exp2 CAI+IVD Complete 
16Ctd (Exp2) vrs Performance 
Classroom(Ctrl) Test on AE 

CAIvrs 
Classroom 

CAI+rVDvrs 
Classroom 

CAI+rVDvrs 
CAI 



72Expl IVD/Hi+AE 
72Exp2 (Exp 1) vrs 
72Ctrl IVD/LofAE 
(Exp2) vrs AE 
only(Ctrl) 

IVD/Hi+AE 
vrs AE only 

IVD/Lo+AE 
vrs AE only 

IVD/Hi+AE 
vrs IVD/Lx)+AE 



Performance 
Test on AE 



(ANOVA) 
na 



.76 
p<.05 



32% 



I.59IC.P 68% 
p<.001 

1.42»c.P 53% 
p<.001 

(ANOVA) 
p<.001 



.56ICP 
p<.05 

.46I.C.P 
ns 

.I2IC.P 
ns 



14% 
11% 

2% 



na Kimberlin 
(1982) 

Also in 

Gibbons, 
Cavagnol, and 
Lines (1982) 



.94 acchinelli, 
(n&OS) KeUer, and 
(1) Hannon 
(1984) 

.84 

(n&OS) Also in 
(2) 

Gcchinelli 
.44 (1984) 
(n&OS) 
(3) 
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Notes Concerning Effect Size: 

I - Included in Effect Size Overview Statistics; T ~ Tutorial Approach only; S - Simulation approach only; 

C - Combined TuK)rial and Simulation; K - Knowledge Outcome; P - Perforaiance Skill Outcome 4 
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Table 1 (Continued). Studies of Interactive Videodisc Instmcrion Effectiveness . 

Comparison Outcome 



Instructional Instructional 
SfittioS Content 



Instructional Disc 
Approach Level N 



Effect Percent Cost 
-iizc Imi2L RatlQ Reference 



Military 
Training 



Electronics 
Maintenance 



Tutorial on 

Simulated 

Equipment 



ni 



25Expl IVD/Hi(Expl) 
26Exp2 vrs IVD/Lo 
23Ctri (Exp2)vrsAE 
only (Cti;l) 

IVD/Hivrs 
AEonly 

IVD/Lo vrs 
AEonly 

IVD/Hivrs 
rVD/Lo 

IVD/Hi(Expl) 
vrs IVD/Lo 
(Exp2) vrs AE 
only (Ctrl) 

IVD/Hivrs 
AEonly 

IVD/Lo vrs 
AEonly 

IVD/Hivrs 
IVD/Lo 

20Exp IVDandAE 
20Ctrl vrs. AE only 



Performance (ANOVA) 
Test on AE ns 



.27i'C'P 8% 
ns 

.16i'C.P 5% 
ns 

.12^'C'P 3% 
ns 

Time to (ANOVA) 
(Complete the ns 
Performance 
Test 

-6% 

ns 

-.23i'C'P -10% 
ns 

ns 

Time to 25% na 

Complete p<.(X)l 
the Training 



Ketner (1984) 



Notes Concerning Effect Size: 

I ~ Included in Effect Size Overview Statistics; T -- Tutorial Approach only; S -- Simulation approach only; 
C - Combined Tutorial and Simulation; K - Knowledge Outcome; P -- Performance Skill Outcome 



Table 1 (Continued). Studies of Inte ractive Videodisc Instruction Effecriveness 



Instnictional Instructional 
2£ltiQg Content 



Mflitary 
Training 



Electronics 
Maintenance 



Instructional 
Approach 



Disc 
Level N 



Comparison Outcome 



Effect Percent Cost 
-Size ImEL Ratio 



Reference 



Simulated HI 
Equipment 



21Exp IVDvrsAE 
22Ctri Practice 



Course 
Knowledge 
Test (Paper) 

Procedures 

Performance 

Test 



I9I.S.K 
ns 

ns 



2% 



4% 



.27 
(H) 

.03 
(OS) 



Pieper, 
Richardson, 
Harmon, and 
Keller (1984) 



Completion -.21^'^'^ -5% 
Time for ns 
Above Test 

Troubleshoot- .93^'^'^ 19% 
ing Perfor- p<.005 
mance Test 



Completion -.08^'^''' -2% 
Time for ns 
Above Test 



Militaiy 
Training 
(Infantry 
Officers) 



Military 
Operations on 
Urbanized 
Terrain 
(MOUT) 



Simulated 
Recon- 
naissance 
Using Sur- 
rogate Travel 



ni ie9Exp IVDvrs 

156Qrl Conventional 
Training 

148Exp 
ISOCtri 



148Exp 
ISOCtrl 



Existing 
MOUT Per- 
formance Test 

MOUT 

Simulation 

Test 

Defense 

Deployment 

Test 



ns 

.74I.S.P 
p<.05 

.40I.S.K 
p<.05 



2% na 



10% 



19% 



King and 
Reeves (1985) 



Notes Concerning Effect Size: 

I ~ Included in Effect Size Overview Statistics; T - Tutorial Approach only; S - Simulation approach only; 
C - Combined Tutorial and Simulation; K - Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Studies of TntCTacrive Videodisc Instmcrion Rffecriveness 



Instructional 
Scttinir 


Instructional 
Content 


Instructional Disc 
Anoroach Level N 


Comparison 


Outcome 


Effect 
Size 


Percent 
impr. 


Cost 
Ratio 


Reference 


Militaiy 
Txaining 


Electrorics 
Maintenance 


Tutorial with III 

Simulated 

Equipment 


48Exp 
SlCtri 


IVD+AEvrs 
AEPtactice 


Paper Test of .2(fi*^^^ 
Knowledge ns 


1% 


na 


Wilkinson 
(1985) 












Performance 
Test on AE 


ns 


5% 
















Time to 
CcHnplete 
Test on AE 


ns 


12% 






Military 
Training 


Sonar 

Maintenance 


Tutorial on III 

Simulated 

Equipment 


8Exp 
SCtrl 


IVDaiidAE 
vrs. AE only 


Test on AE 
Procedure I 

Time to 
Complete 
Procedure I 


29I.C.P 
p<.05 

p<.001 


39% 
40% 


See next WiUiams and 
reference Harold (1985) 










• 


Test on AE 
Procedure n 


na 
ns 


33% 
















Time to 
Complete 
Procedure II 


p<.001 


28% 






Sanie 


Sanie 


Same Same na 


Same 


15 year life- 
cycle costs 


na 


na 


.05 

(Hand 
OS) 


Green, Beger, 
and Dunlap 
(1986) 



''y2 

Notes CcMiceming Effect Size: 

I -- Included in Effect Size Overview Statistics; T Tutorial Approach only; S - Simulation approach only; 
C - Combined Tutwial and Simulation; K -- Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Studies of Interactive Virteodis c Instruction Effecrivem^^^ 



Instnictional Instructional 
Setting Content 



Militaiy 
Training 



Interpersonal 
SkiUs 



Instructional Disc 
A pproach Level N 



Simulated Ul 
Inteipersonal 
Situations 
with TutCHial 
Feedback 



30Expl 
SlCtrll 



S:smmn'£m Outcome 



EfTcct 



!VD(EiCp)vrs Content Testl ANOVA 



29Clri2 


vrs Programined Abuse) 
Tcxt(Ctil2) 


30Exp 


rVDvis 
Role Play 


53I.C.K 
ns 


30Exp 


IVDvrs 
Prog. Text 


31I.C.K 
ns 




IVDvrs 
Role Play vrs 
Prog. Text 


Content Test2 ANOVA 
(Taking ns 
Charge) 




IVDvrs 
Role Play 


ns 




IVDvrs 

nug. IcXl 


ns 




rVDvrs 
Role Play vis 
Prog. Text 


Content Tests ANOVA 
(Meeting the ns 
Troops) 




IVDvrs 
Role Play 


ns 




IVDvrs 
Prog. Text 


05I.C.K 
ns 



Percent Cost 
Impr. Ratio 



Reference 



19% 



10% 



hi Schroeder, 

Dyer, Czemy, 
Youngling, 
and Gillotti 
(1986) 

Also 

Schroeder, 
Hall, and 
Morey (1985) 



25% 



29% 



■11% 



1% 




Notes Concemi/ig Effect Size: 

I - Included in Effect Size Overview Statistics; T -- Tutorial Approach only; S -- Simulation approach only; 
C ~ Combined Tutorial and Simulation; K -- Knowledge Outcome; P -- Perfwmance Skill Outcome 
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Table 1 (Contini'td). Studies of Inte ractive Videndisc Instnicrion Effecriveness . 



Instructional Instructional Instructional Disc Effect Percent Cost 

Contcnl ABPiaach Uvd N Comparison Qiircor t Size Impr. Ratio Reference 

IVDvrs Content Test4 ANOVA 

Role Play vrs (Perfamance ns 

Prog. Text Counseling) 

IVDvrs -.n^-^'^ -6% 

Role Play ns 

rVDvis .74i«c.K 33% 

Prog. Text ns 

IVDvrs Content TfistS ANOVA 

Role Play vrs (L'lsubordina- p<.05 

Prog, '^ext tion) 

IVDvrs 1.12i«c.K 43% 

Role Play p<.05 

IVDvrs .97I.C.K 3g% 

Prog. Text p<.05 

IVDvrs Content Teste ANOVA 

Role Play vrs (Personal ns 

Prog. Text Crisis) 

IVDvrs -.45i'C.K .12% 

Role Play ns 

IVDvrs -.16^'^'^ -6% 

Prog. Text ns 



Notes Concerning Effect Size: 




I -- Included in Effect Size Overview Statistics; T -- Tutorial Approach only; S - Simulation approach only; 
C - Combined Tutonal and Simulation; K - Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Smdies of Interactive Videti disc Tnstnicrion Effecriveness . 



Instructional Instructional 
Setting Content 



Militaiy 
'Training 



Industrial 
Training 



Instructional Disc 
.^BPIOach Level N 



Comparison Outcome 



Effect Percent Cost 
Size Impr. Ratio 



Reference 



Helicopter 

Maintenance 

(Hydraulics) 



CPRReccrtifi- 
cation Training 
{<x Registered 
Nurses 



Tutorial (xi 

SimuliUed 

Equipment 



ni 



Simulaticxi III 



36Exp 
35Ctri 



ISExp 
14Ciri 



14Exp 
14Ctrl 



ISExp 
14Ctii 



14Exp 
13Ctrl 



Compulsoy Paper Test of .48^'^''^ 
Independent of Course p<.OS 
Study with vrs Knowledge 
without IVD 

Performance .14^*^*'* 
Test Using ns 
Training 
Device 



rVDvrs 
Lecture 
(Acute Care) 

IVD vrs 
Lecture 
(Non-Acute Care) 

rVDvrs 
Lecture 
(Acute Care) 

IVD vrs 
Lecture 

(Non-Acute Care) 



Knowledge 
Test 



Knov/iedge 
Test 



Skill Test 



Skill Test 



.I.I4I.S.K 
ns 

.OOI.S.K 
ns 

..O7I.S.P 
ns 

.07I.S.P 
ns 



4% 



2% 



■4% 



0% 



■1% 



1% 



na 



Malecand 

Luszczak 

(1987) 



.61 

(OS) 



Aukerman 
(1986) 



Industrial Communication Tutorial with III 
Training Circuit Equipment 
Maintenance Simulation 



lOExpl IVD(Expl)vrs 
10Exp2 Classroom(Exp2) 
12Ctrl vrsOn-JobTrag 
(Ctrl) 



na 



North (1988) 



Notes Concerning Effect Size: 

I ~ Included in Effect Size Overview Statistics; T - Tutorial Approach only; S ~ Simulation approach only; 
C ~ Combined Tutorial and Simulation; K - Knowledge Outcome; P ~ Performance Skill Outcome 



Table 1 (Continued). Studies of Interactive Vide odisc Tnstmcrion Effectiveness 



Instructional Instructional Instructional Disc 
Sfilting Content APDioach Level N 



Industrial 
lYaining 



Comparison Outcome 



Effect Percent Cost 
-Size Imi2L Eado Reference 



Handling of 

Hazardous 

Materials 



Tutorial 



ni 



IVDvrs 
Classroom 

IVDvrs 
On-Job Tmg 

Gassroom vrs 
On-Job Tmg 

105Exp IVDvrs 
104Ctrl Videotape 



Equipment 
Knowledge 



J5I.C.K 
p<.05 

1.42I.C.K 
p<.001 

.77 
p<.05 



20% 



58% 



31% 



Errors on I.IO^--'^ 47% 
Content Test P<.001 



na Bosco and 

Wagner (1988) 



Higher Biology Tutorial 
Educaticn "Development 

of Living 

Things" 



m 25Exp IVDvrs 
24Ctri Lecture 
(Student 
Volunteers) 



Biology 
Knowledge 
(Objective 
Test) 



.27I.T.K 
ns 



8% na 



Biology .93^''^'^ 
Knowledge p<.05 
(Short Answer 
Test) 



IWk 
Retention 
(Objective 
TfcSt) 

IWk 
Retention 
(Short Answer 
Test) 



29I.T.K 
ns 



ns 



73% 



6% 



36% 



Bunderson, 
Baillb, Olsen; 
Lipson, and 
Rsher(1984) 

Also in 

Bunderson, 
Olsen, and 
Baillio(1981) 



Notes Concerning Effect Size: 

I ~ Included in Effect Size Overview Statistics; T - Tutorial Approach only; S - Simulation approach on)y; 
C ~ Combined Tutorial and Simulation; K - Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Studies of Int eractive Videodisc Instruction Effectiveness . 

Comparison Outcome 



Instructional Instnictional 
Smog Content 



Instructional Disc 
.^Epmacb Level N 



Efifect Percent Cost 

Imi2L Ratio Reference 



28Exp 
25Ctri 



24Exp 
73Ctri 



IVDvrs 

Lecture 

(Utah) 

(Rand(Mii 

Assignment) 



IVDvrs 

Lecture 

(Texas) 

(Rand(Mn 

Assignment) 



Learning 
Time 

Biology 
Knowledge 
(Objective 
Test) 

Biology 
Knowledge 
(Short Answer 
Test) 

Learning 
Time 

Biology 
Knowledge 
(Objective 
Test) 

Biology 
Knowledge 
(Short Answer 
Test) 

Learning 
Time 



ns 

p<.05 



.58'-T.K 
p<.05 



p<.05 

.45I.T.K 
p<.05 



59i.T,K 
p<.05 



p<.05 



32% 



18% na 



37% 



44% 



6% 



na 



17% 



32% 



ERIC 



Notes Concerning Effect Size: 

I - Included in Effect Size Overview Statistics; T - Tutorial Approach only; S ~ Simulation approach only; 

C - Combined Tutorial and Simulation; K -- Knowledge Outcome; P - Performance Skill Outcome , . , . 
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Table 1 (Continued). Snidies of Interactive Vide odisc Instruction Effecriveness . 



Instructional 
Setting 



Instructional 
Content 



Instructional Disc 
AotMoach Level N 



Comparison Outcome 



Effect Percent Cost 

Size Jmpr. Ratio Reference 



Higher 
Education 



Fbreign 
Language 
(Beginning 
French) 



Tiitoial 



LI 26Expl IVD(Expl)vrs Course (ANOVA) 
26Exp2 aassnx)m(Exp2) Knowledge p<.(X)l 
26Ctri YTS No Instruction (Multiple 



(Ctrl) 

IVDvrs 
Classroom 

IVDvrs 

No Instruction 

Classroom vrs 
No Instruction 

IVDvrs 
Qassroom vrs 
No Instruction 



IVDvrs 
Classroom 

IVDvrs 

No Instruction 

Qassroom vrs 
No Instioiction 



Choice Items) 



Course 
Knowledge 
(Completion 
Items) 



09I.T.K 
ns 

1.50 
p<.05 

1.40 
p<.05 

(ANOVA) 
p<.001 



1.02^'T.K 
p<.05 

2.52 
p<.05 

1.20 
p<.05 



na 



Crotty (1984) 



2% 



31% 



29% 



64% 



294% 



139% 



Notes Cbnceming Effect Size: 

I - Included in Effect Size Overview Statistics; T - Tutorial Approach only; S -- Simulation approach only; 
C - Combined Tutorial and Simulation; K - Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Studies of Interactive Videod isc Instnicrion Effecriveness . 

Comparison Outcome 



Instnictional InstnKtional 
Setting Content 



Higher 
Education 



Higher 
Education 



Higher 
Education 



Higher 
Education 



Instructional Disc 
Approach Level N 



Physics Simulated ni 

"Puzzle of the Laboratcxy 
Tacoma Bridge 
Collapse" 



Physics Simulated n 

"Puzzle of the Laboratmy 
Tacoma Bridge 
Collapse" 

Biology Simulated ni 

(Respiration) Laboratmy 



Biology 
(OinTjate and 
Life) 



Public Health 

(Smokeless 

Tobacco) 



TutcMial 



ni 



21Exp 
16Ctri 

22Exp 
19Ctri 

18Exp 
ISCtrl 



22Exp 
8Ctri 



IVDvrs 

Laboratory 

Equipment 



IVDvrs 

Labnatoiy 

Equipment 



IVDLabvrs 

Traditional 

Lab 



22Exp Same 
25Ctrl 



Effect Percent Cost 
-Size ImpL Ratio. 



Reference 



Physics 
Knowledge 

Scientific 
Attitudes 

Physics 
Knowledge 
(Standing 
Waves) 

Content 
Test (Paper) 



Same 



42Expl IVD(Expl) vrs Knowledge 
61Exp2 Individual Tape Test 
43Exp3 (Exp2) vrs Group 
59Ctrl Tape(Exp3) vrs 

No Instruction (Ctrl) 

IVDvrs 
Individual Tape 

IVDvrs 
Group Tape 



.29I.S.K 11% na 
ns 



.24I.S.K 
ns 

-.17 
ns 



2% 

-4% na 



-M^'^'^ -8% 4.49 
ns (H) 



23% na 



.83I.S.K 
p<.01 



(ANOVA) 
p<.001) 



p<.01 

1.32'''''''^ 32% 
p<.01 



.02 
(OS) 



na 



Stevens (1984) 



Wankel (1984) 



Davis (1985) 



Levenson, 
Morrow, and 
Signer (1985) 



ERIC 



Notes Concerning Effect Size: 

I - li»nluded in Effect Size Overview Statistics; T Tutorial Approach only; S - Simulation approach only; 
C " Ccmbined Tutorial and Simulation; K -- Knowledge Outcome; P ~ Performance Skill Outcome 
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Table 1 (Continued). Smdies of Interactive Vide odisc Instruction KfTticriveness 



Instructional 
Setting 


Jiistructional 
Content 


Instructional Disc 
Aooroach Level N 


Comparison 


Outcome 


Effect 
Size 


Percent 
Iir.pr. 


Cost 
Ratio 










IVDvrs 

No Instruction 




, 66 

T COl 


104% 












Individual Tape vrs 
Group Tape 


.50 
p<.01 


12% 












Individual Tape vrs 
No Instruction 


1.86 
p<.01 


73% 












Group Tape vrs 
No Instniction 




1.38 
p<.01 


54% 




Higher 
Education 


Chemistry 


TutOTialwith II 
Simulated 
L ^boratory 


58Exp 
54Ctrl 

57Exp 
55Ctrl 


IVDvrs 
Videotape 


Course 
Knowledge 

Graphical 
Antdysis 


47I.C.K 
p<.05 

30I.C.K 
ns 


56% 
10% 


na 


Higher 
Education 


Chemistry 
(Kinetics and 
Equilibrium) 


Tut(xialwith III 

Simulated 

Laboratc^ 


26Exp 
23Ctrl 


IVD+Lab 
vrs Lab only 


Chemistry 
Knowledge 
(Lab Reports) 


59I.C.K 
p<.05 


17% 


na 








21Gxpl 
17Exp2 
49Ctrl 


IVD(Expl) vrs Course 
IVD+Lab (Exp2) Knowledge 
vrsLabop\ ^Ctrl) 


(ANOVA) 
p<.001 




na 










IVDvrs 
Lab only 




1.08J'C.K 
p<.05 


40% 












rVD+Ub 
vrs Lab only 




94I.C.K 
p<.01 


34% 





Reference 



Russell, 
StPiSkun, and 
Mtchell 
(1985) 



Smith, Jones, 
and Waugh 
(1986) 



Notes Concerning Effect Size: 

I - Included in Effect Size Overview Statistics; T -- Tutorial Approach only; S -- Simulation approach cnly; 
C -- Combined Tutorial and Simulation; K - Knowledge Outcome; P - Performance Skill Outcome 
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Table 1 (Continued). Studies of Interactive Videodisc Instnicrion Effecriveness. 



Instructional Instructional 
Setting rontent 



Instructional Disc 
Approach Level N 



Comparison Ontrome 



Effect Percent Cost 
Size Tmpr. Ratio 



Reference 



Higher 

Educaticm 



Higher 
Education 



Chemistry 
(Kinetics and 
Equilibrium) 



Tut(malwiti) III 

Simulated 

Laboratory 



Public Healti) Simulation ni 
(CPR 

Instructior^ 



IVDvrs 
IVD+Lab 

25Exp IVD vrs 
27Ctrl Lab 



48Exp IVD vrs 
SlCtrl Classroom 
Instruction 



Course 
Knowledge 



Course 
Knowledge 

Performance 

(Single 

Rescuer) 



.I9I.C.K 
ns 

.66I.C.K 
p<.05 

ns 

.39I.S.P 
ns 



Performance .23''^''' 
(Two Rescuer) ns 

Performance .4l''^''' 

(Obsnncted p<.05 
Airway #1) 

Performance .SO'-^-'' 

(Obstiucted p<.05 
Airway #2) 

Performance .67''^''' 

(Obsnncted p<.05 
Airway #3) 



22% na 

2% na 
47% 

19% 
23% 

63% 

59% 



Jones (1987) 



Lyness(1987) 



ERIC 



Notes Concerning Effect Size: 

I -- Included in Effect Size Overview Statistics; T -- Tutorial Approach only; S -- Simulation approach only; 
C -- Combined Tutorial and Simulation; K Knowledge Outcome; P -- Performance Skill Outcome 
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Tabic 1 (Continued). Studies of Interactive Video disc Instruction Effecriveness 



Instructional Insductional 
SsiSm Content 



Instructional Disc 
Apprpagh Level N 



Comparison Outcome 



Effect Percent Cost 

S'm Impr. Ratio Reference 



Performance 
(Infant) 



ns 



-23% 



Performance -.Sl"-^-^ -12% 
(Infant 
" ostructed 
Airway #1) 



ns 



Higher 
Education 



Foreign 
Language 
(Beginning 
Spanish) 



Tutorial 



ni 



Performance 
(Infant 
Obstructed 
Airway #2) 

23Expl IVD linear Course 
23Exp2 (Expl)vrsIVD Knowledge 
23Exp3 Segmented 
23Ctil (Exp2) vrs IVD 

Interactive (Exp3) 

vrs Irrelevant bis- 

oiiction(Ctil) 

IVD Interactive 
vrs IVD Segmented 

IVD Interactive 
vrs rVD Linear 

IVD Interactive 
vrs Irrelevant Inst. 

IVD Segmented 
vrs IVD Linear 



ns 



(ANOVA) 
p<.001 



l_4ll.T.K 
p<.001 

2.49I.T.K 
p<.001 

3.84 
p<.01 

ns 



6% 



na 



Verano 
(1987) 



33% 



58% 



88% 



18% 



Notes Concerning Effect Size: 



I -- Included in Effect Size Overview Statistics; T - Tutorial Approach only; S -- Simulation approach only; 
C -- Combined Tutorial and Simulation; K -- Knowledge Outcome; P -- Performance Skill Outcome 
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Table 1 (Condnued). Studies of Interactive Videndisc Tnstnirrinn Fffectiveness 



tostnictionai Instructional Instructional Disc Effect Percent Cost 

^^ontSDl Approach LessLIii QQmymm Outcome Size Impr Ratio Rftfen>nrP 



IVD Segmented 
vrs Inelevant Inst. 


1.80 
p<.05 


41% 


IVD Linear vrs 

IXTCICValll lllau 


.83 
ns 


19% 


IVD Interactive Course 
vrs rVD Knowledge 
Segajciited (4 Wk Delay) 


79I.T.K 
p<.001 


30% 


rVD Interactive 
vrs IVD Linear 


2 24I.T.K 
p<.001 


56% 


IVD Segmented 
vrs IVD Linear 


.82i'T.K 
ns 


21% 


IVD Inlcractivf 
vrs Irrelevant ]nst. 


2.06 
p<.001 


51% 


IVD Segmented 
vrs Irrelevant Inst. 


.68 
ns 


17% 


IVD Linear vrs 
Irrelevant Inst 


-.12 
ns 


-3% 



ERIC 



Notes Concerning Effect Size: 

I - Included in Effect Size Overview Statistics; T - Tutorial Approach only; S ~ Simulation approach only; 
C - Combined Tutonal and Simulation; K - Knowledge Outcome; P - Performance SkiH Outcome 
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Tabic 1 (Coniinued). Studies of Interactive VidenHi sc Instnicrion Fffecriveness . 



Instructional Instructional 
Ssmog Content 



Higher 
Education 



Instructional Disc 
Approach Level N 



Comparison Outcome 



Effect Percent Cost 
-Size Impr. Ratio Rffgiypyf 



Chemistry Tutorial widi HI 26Exp IVDvrs 
(Gas Analysis) Simulated 22Ctrl Lab 

Laboratory 



Spectrometer l.OS^'^''' 69% na 
Usage -Score p<.01 
(3 Wk Delay) 

Spectrometer 39% 
Usage -Time p<.01 
to Complete 
(3 Wk Delay) 



Knowledge 
Test(2Wk 
Delay) 



51I.C.K 
pcOl 



10% 



Jones (1988) 




Notes Concerning Effect Size: 

I ~ Included in Effect Size Overview Statistics; T - Tutorial Approach only; S Simulation approach only; 
L - Combmed Tutorial and Simulation; K - Knowledge Outcome; P ~ Performance Skill Outcome 
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Tabic 2. Average Effect Sizes f<x Tutorial and Simulation Instruction, Knowledge and 
Peifoniiance Outcomes. 



Knowledge Performance 



QwffPmgS Outcomes Totals 

Tutorial .97 .97 

Approaches (N = 22) (N= 0) (N = 22) 

Simulation .05 .19 .14 

Approaches (N=ll) (N=18) (N = 29) 

Combined .49 .55 .52 

Approaches (N = 24) (N = 20) (N = 44) 



Totals .59 .38 .51 

(N = 57) (N=38) (N = 95) 



ERIC 
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EFFECTIVENESS OF INTERACTIVE VIDEODISC 
IN ARMY COMMUNICATIONS TRAINING 



Executive Summary 

The RAND Corporation 
January 1989 

As the technical sophistication of military weapon and support 
systems has increased, the services have sought new ways to use 
technology to train for more complex tasks. Prominent among new 
training techno log ies is Interactive Videodisc (IVD)i which links a 
microcomputer and laser videodisc to provide interactive instruction 
with high-resolution video displays. This report documents two RAND 
studies of Army IVD applications, employing rigorous experimental 
designs and post-experimental performance assessments to evaluate the 
effects of alternative uses of IVD in Army communications training. 
RAND report. 

BACKGROUND 

Defense modernization has brought complex hpw weapon and support 
systems into the inventories of the military services. Improvements in 
military technology, however, bri-^g conflicting pressures on the 
services' training establishments. The growing variety and complexity 
of many new systems tend to raise skill requirements, leading to 
pressures for longer and more expensive trainin;^ courses. At the same 
time, some operational equipment has become so costly that the training 
base can at best afford only a few pieces that resemble those actually 
used in the field. In field units themselves, where equipment is 
available, it is difficult to assure standardization and quality of 
training. 
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Military trainers have begun to respond to such challenges by 
expanding their use of new computer-based, visually-oriented training 
devices and simulators. These technologies have the potential to 
simulate a variety of new equipment, provide individualized yet 
standardized instruction, engage learners in dynamic problem-solving 
situations, and provide immediate feedback about performance. Among 
recent innovations, interactive videodisc technology, which consists of 
an integrated microcomputer, video display, laser videodisc, and 
instructional software (termed interactive courseware), represents a new 
training device with considerable promise. 

The U.S. Army Signal Center at Fort Gordon, Georgia, has pioneered 
the use of IVD systems for training soldiers in a variety of 
communications-electronics military occupational specialties (MOSs). 
Signal Center developers of an early IVD system a predecessor to the 
Army's Electronic Information Delivery System, which is being acquired 
in large numbers starting in Fiscal 1988 -- have hypothesized that 
school-based IVD training may increase student proficiency and reduce 
hands-on training requirements for a broad range of specialties. They 
also believe that IVD systems have potential in field units for 
refresher and on-the-job training. Demonstration of these hypothesized 
benefits could affect decisions by the Army and by the other services 
about purchasing EIDS hardware, developing interactive courseware, and 
allocating EIDS training systems across various specialties and 
environments . 

The possible benefits of IVD technology are of interest not only to 
the services, but also to the Office of the Secretary of Defense (OSD), 
which has oversight responsibility for the efficiency of training. OSD 
interest in the Signal Center experience, and in the effectiveness of 
IVD more broadly, has been heightened the various services have 
become interested in applications of interactive videodisc technology. 
However, to date the systematic data needed to assess the potential 
benefits of IVD have been lacking. To provide such data and to 
establish a model for future research in this area, RAND undertook a 
series of studies of IVD in cooperation with the Signal Center and OSD's 
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Defense Training and Performance Data Center. This report presents the 
resul:^ v:f these studies and their implications for uses of IVD in 
military settings. 

OBJECTIVES AND APPROACH 

The objectives of this study were to develop a methodology for 
assessing the benefits of innovative training technologies, to apply 
this methodology to evaluate the effectiveness of an IVD training system 
used at the Army Signal Center, and to define general conditions for 
effective use of IVD technology. 

Our approach applied principles of controlled experimentation to 
compare effects of alternative methr^ds used to train equivalent groups 
of soldiers. We report the results of two studies. In both, the 
effects of traditional hands-on equipment training (the control 
condition) are compared with effects of a trainxng regimen using IVD 
(the experimental condition). The experimental and control groups were 
formed using a statistical randomization model developed at RAND which 
provides a close match between groups on such factors as aptitude, 
educational background, demographic characteristics, and military 
expcrierice. The training received by each group was carefully 
monitored, and the effects of alternative training methods were compared 
using multivariate analysis ol objective, training- and job-related 
ptrformance criteria. 

The two studies examine the two mos"" common applications of IVD in 
the Army: As a device used to supplement or augment existing hands- 
on training, and as a device used simulate or replace hands-on equipment 
training. The first use of IVD increases training opportunity, while 
increasing costs; the second use of IVD can maintain existing training 
opportunity, while decreasing costs. These studies provide empirical 
evidence of IVD effectiveness in the specific MOSs trained, and they 
point to implications for IVD training policy in many other military 
settings. 



A-3 

81 



SUPPLEMENTARY TRAINING WITH IVD: MOS 31M 

The initial experiment evaluated the effects of IVD on student 
proficiency when used as a device to supplement hands-on equipment 
training in MOS 31M, Multichannel Communications Equipment Operator. 
The experimental training took place during two weeks of the course when 
students learned to install "low-capacity" radio equipment (AN/T^C-145) . 
The experiment lasted seven months and covered 428 active-duty trainees 
who were assigned to onu of two groups. The control group received 
hands-on training at installation using only radio assemblages, while 
the experimental group received both hands-on training with radio 
assemblages and IVD training. Each group had an equal number of radio 
assemblages available (normally 10 assemblages for a class of up to 25 
students). In the experimental group, the IVD provided an additional 
eight training positions to help make up the difference between the 
numbers of trainees and assemblages. 

Several weeks later, tlie performance of each trainee at assemblage 
installation was assessed using the Reactive Electronic Equipment 
Simulator (REES) , a high-fidelity, computer-controlled facility that 
contained the pertinent radio assemblages. The REES computer provided 
data on the accuracy with which trainees accomplished the installation, 
as well as the amount of time and effort required to successfully 
install the radio assemblage. Traine'is* job knowledge was also assessed 
U5iing a written examination, which contained elements of job knowledge 
that were trained, as well as measures of trainees* attitudes toward the 
training that they received. 

The research hypothesis in this study was that IVD use would 
increase the efficiency of training, while improving student 
proficiency. Results showed that the IVD was extensively implemented in 
the experimental classrooms; the addition of IVD to the classroom led to 
a 45 percent increase in time spent practicing installation of radio 
assemblages. Thus, these students received increased training 
opportunity without lengthening their overall amount of time in the 
course. In this respect, the use of IVD allowed instructo'-s to make 
more efficient use of student time. 



IVD training also increased soldier proficiency, as assessed in the 
high-fidelity siipulator. Regression analyses showed that supplemental 
IVD training caused statistically significant reductions in the time 
needed to install the radio equipment, the number of trials (amount of 
effort) needed to accomplish the installation, and the likelihood of a 
student error during the installation process. These reductions were 
modes i, however, ranging between 10 and 20 percent. 

SUBSTITUTION TRAINING WITH IVD: MOS 31Q 

The second experiment examined the effects of substituting IVD 
technology for more expensive equipment in MOS 31Q, Tactical 
Satellite/Microwave Systems Operator. This experiment, lasting ten 
months and encompassing 336 trainees, focused on training the alignment 
and adjustment cf c implex and expensive tropospheric scatter (TROPO) 
radio assemblages. Tlie approach held the imount of training opportunity 
constant, while varying the resources used for training. Students were 
assigned to one of two groups: Half carried out exercises in a 
classroom equipped with 7 TROPO radios and 8 closely-related line-of- 
sight (LOS) radios, while the other half carried out similar exercises 
ii. a classroom that contained only one of each type of radios but had 
eigh*: IVD units a much less ej'pensivc complement of training devices. 

Immediately after the trainirg, we assessed the performance of each 
trainee using a hands-on test based on the Army Soldier's Manual, 
including three relevant tasks (IF gain and AGC alignmer*:, and squelch 
adjustment). The hands-on tests were administered by objective 
assessors, trained and monitored by RAND, who were unaware of how each 
soldier had been trained. For eac'i test, we determined whether the 
trainee could accomplish each of the tasks within the respective Army 
time standard, and we recorded errors made during task performance. 
Trainees also received a written test providing measures of task 
knowledge and attitudes toward the training that taey received. 

For this sttdy the research hypothesis was that students would be 
equally proficient at the tasks, whether they wero trained under the 
traditional equipment -only regimen or under the alternative regimen in 



which IVD was used at a substantial saving in training resources. Our 
analyses confirm the hypothesis for measures of proficiency and job 
knowledge. As an illustration, we summarize results for performance at 
the IF Gain alignment, the most difficult of the tasks. The results 
show that students used IVD extensively in the experimental classroom, 
accomplishing 58 percent of their training sessions on IVD. Students in 
the control group received approximately the same number of training 
sessions, but of course 100 percent of their training was done on actual 
equipment. Despite this substitution, the performance of the groups on 
the hands-on test was statistically indistinguishable. 

Our analyses show similar results for student performance in AGC 
alignment and squelch adjustment: Ability to accomplish the task was 
the same, whether students were trained with actu il equipment or with a 
mix of IVD and actual equipment. However, for these tasks the 
IVD-trained students appeared slightly more likely to make procedural 
errors, and they were less satisfied with the training they received. 

CONCLUSIONS 

The results of these experimental studies show that IVD technology 
can be beneficial in its two most common type? of application: as a 
supplement to existing training or as a substitute for more expensive 
training resources. In MOS 31M, the addition of IVD provided increased 
training opportunity and caused improvements in measures of subsequent 
task proficiency*. In MOS 31Q, the replacement of some equipment 
training with IVD training did not diminish students' ability to perform 
the relevant tasks. These studies thus confirm hypothesized benefits of 
IVD technology for the applications that we examined. 

At the same time, however, information collected in h-"h studies 
suggested some important conditions that may affect when and where one 
choose^ to use IVD. In MOS 31M, our data showed that most trainees 
received ample hands-on training opportunity, even in the control group 
where instructors perceived an equipment shortage; in fact, nearly all 

* Other research studies suggest that the additional practice 
provided by such training technologies can permit a reduction in the 
allotted training time. 



trainees were eventually able to perform the installation successfully. 
We speculated (though without strong statistical evidence) that frequent 
practice on real equipment had given most students a fairly high level 
of basic proficiency, which may have limited the benefits that could be 
gained by adding IVD. If correct, this suggests an important criterion 
for using IVD as a supplement to existing training resources; 
Supplementation is most likely to pay off in those situations where 
opportunities to train are mere scarce, the difficulty of the task is 
more demanding, and existing proficiency is more unsatisfactory. 

The 31Q experiment confirmed that substituting IVD in place of 
h£nds-on training is an attractive and feasible method of reducing 
equipment costs. However, there are likely to be limitations to such 
substitution, and the 31Q experience points to them. Even though the 
IVD-trained students in the 31Q study were equal' y capable of 
accomplishing their tasks, they were slightly more likely to make 
certain procedural errors, and they expressed less satisfaction with 
training. We believe that these differences may arise from the extreme 
contrast in hands-on opportunity experienced in the two groups; the 
equipment -trained students enjoyed ample practice on real radio 
assemblages, while the IVD-trained group had only brief exposure to 
actual equipment. If true, this suggests that certain minimum levels of 
hands-on training may be required to ensure competency and self- 
confxdencd among trainees. 

Thus the results of both experiments indicate that IVD can be an 
effective element of training, and they point to some conditions for 
using the technology wisely. Given the costs of acquiring IVD systems 
and developing supporting interactive courseware, however, its 
application seems best encouraged in those circumstances where it can be 
used to greatest potential: To save training costs as part of a 
training resource mix, or to provide additional training where 
improvements in proficiency are needed and the additional costs can be 
justified. 
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ORGANIZATIONS CONTACTED 



ABC Technologies International, Inc, 
805 DuPont Street 
Suite ; 

Bellinsiam, WA 98225 

Access Network 
295 Midpark Way, S.E, 
Calgary, Alberta 
Canada T2X 2A8 

Actronics 

810 River Avenue 

Pittsburgh, PA 15212 

Advanced Technology, Inc. 
12005 Sunrise Valley Drive 
Reston, VA 22091 

Air Force Communication Command 
1872 SCHS/TUV 

Keesler Air Force Base, MS ^534-6340 

Air Force Human Resource Laboratory/MOMJ 
Brooks Air Force Base, TX 78235-5601 

Air Training Command 
HQ ATC XPCTR 

Randolph Air Force Base, TX 78150 

Air Training Command 

3300 Technical Training Wing 

Keesler Air Fore* Base, MS 39534-5000 

Alamo Learning Systems 
37000 Grand River 
Farmington Hills, MI 48024 

The Alive Center 
1248 Weathervane Lane 
Akron, OH 44313 
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Allen Communication 
140 Lakeside Plaza II 
5225 Wiley Post Way 
Salt Lake City, UT 84116 

Alta Association 
436 E. 3400 N. 
Logan, UT 84321 

American Language Academy 
11426 Rockvllle Pike, Suite 200 
Rockvllle, MD 20852 

American Journal of Nursing Company 
555 W. 57th Street 
New York, NY 10019 

American Society for Training and Development 
1630 Duke Street 
P.O. Box 1443 
Alexandria, VA 22313 

Andersen Consulting 
33 W. Monroe 
Chicago, II 60603 

Andersen Consulting 

901 Main Street - Suite 5600 

Dallas, TX 75202 

Annenberg/Corporatlon for Public. 

^^roadcastlng Project 
1111 Sixteenth Street, N.W. 
Washington, DC 20036 

Apple Computer 
Multi-Media Group 
3220 Sacramento 
San Francisco, CA 94111 

Applied Interactive Technologies, Inc. 
621 Lakeland East Drive 
Jackson, MS 39208 
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Applied Learning International 
(Deltak, Advanced Systems, Inc.) 
EAST-WEST Technological Center 
1751 West Diehl Road 
Naperville, IL 60540-9075 

Applied Science Associates 
Metro II, Suite 900 
8201 Corporate Drive 
Landover, MD 20785 

Army Research Institute 
Training Research Laboratory 
5001 Eisenhower Avenue 
Alexandria, VA 22314 

Army Research Institute - Monterey Field Unit 
P,0, Box 5787 

Presidio of Monterey, CA 93944 

Arthur And'* r sen and Company 
1666 K Street, N.W. 
Washington, DC 20006 

Arthur Andersen and Company 
Research and Development 
1405 N, Fifth Avenue 
St. Charles, IL 60174 

Assessment Designs International 
2500 Maitland Center Parkway 
Maitland, FL 32751 

Association of Developers of Computer-Based 

Instruction 
International Headquarters 
Miller Hall 409 
Western Washington University 
Bellingham, WA 98225 

AT&T Corporate Training 
100 Southgate Parkway 
Morristown, NJ 07960 

Bain and Company 
2 Copley Place 
Boston, MA 02116 
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Baker Vldecactlve 
1501 Walnut Street 
Philadelphia, PA 19102 

BBC Elstree Center 
Clarendon Road 
Borehamwood 
Herts WD6 165 
U.K. 

Beckwlth Custom Communications Company 

147 Bell Street 

Chagrin Falls, OH 44022 

Bethlehem Steel Corporation 
Videodisc Team 
Room 1268 Martin Tower 
Bethlehem, FA 18016 

Bloomsburg University 
Department of Mathematics 
McCormlck Building 
Bloomsburg, PA 17815 

BNA Communications, Inc. 
9439 Key West Avenue 
Rockvllle, MD 20850-3396 

Boeing Military Airplane 
P.O. Box 1470 
Huntsvllle, AL 35807 

Boeing Military Airplane 
P.O. Box 7730 
Wichita, KS 67277 

Brighton Polytechnic 

Telsoft Interactive Video Project 

Falmer Brighton BNl 9PH 

U.K. 

Brown University 
Department of Anthropology 
Providence, RI 02912 



B-4 

GO 



Butler-Ralla and Company 
298A Highland Avenue 
Somervllle, MA 02144 

California State Polytechnic University 
Department of Management and Human Resources 
3801 W, Temple Avenue 
Pomona, CA 91768 

California State University - Chico 
Communication Design 
Chico, CA 95929-0504 

California State University - Sacramento 
Division of Nursing 
6000 "J" Street 
Sacramento, CA 95819 

Carnegie-Mellon U.ilversity 
Software Engineering Institute 
Pittsburgh, PA v5213 

Caterpillar 
P,0, Box 787 
York, PA 17405 

Catharon Productions Inc. 
Route 1, Box 8 
Ghent, NY 12075 

CEIT Systems, Inc. 
225 Charcot Avenue 
San Jose, CA 95131 

Central Intel ligf^nce Agency 
Office of Training and Education 
ITD/CBT 

Washington s DC 20505 

Central Piedmont Community College 
"Project Ready" 
P.O. Box 35009 
Charlotte, NC 28235 

Children's Television Workshop 

1 Lincoln Plaza 

New York, New York 10023 
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Cincinnati Milacron Marketing Company 
(CinTraining Training Systems) 
4701 Marburg Avenue 
Cincinnati, OH 45209 

Clemson University 

Agricultural Engineering Department 

115 McAdams Hall 

Clemson, SC 29634-0357 

College of Aeronautics 
La Guardia Airport 
Flushing, NY 11371 

Combustion Engineering 
1000 Prospect Hill Road 
Windsor, CT 06095 

Commission on Peace Officers Standards 

and Training 
1601 Alhambra Blvd. 
Sacramento, CA 95816-7083 

Competence Assurance Systems 
University Park at M.I.T. 
26 Landsdowne Street, Suite 500 
Cambridge, MA 02139-4234 

Computer Sciences Corporation 
813 Diligence Drive 
Suite 110 

Newport Nevo, VA 23606 

Comsell, Inc. 
500 Tech Parkway 
Atlanta, GA 30313 

Conceptual Systems, Inc. 
1100 Wayne Avenue, 12th Floor 
Silver Spring, MD 20910 

Convergent Technologies Associates 

97 Devonshire Drive 

New Hyde Park, NY 11040 

Corporate Resource Associates, Inc. 
333 Twin Dolphin Drive, Suite 225 
Redwood City, CA 94065 
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Crawford Communications 
500 Plasters Avenue 
Atlanta, GA 30324 

Creative Video/Floyd Design 
1465 Northslde Drive - Suite 110 
Atlanta, GA 30318 

Defense Intelligence College 

Research Center 

DIA/DIC-R 

Washington, DC 20340-5485 

Defense Language Institute 
Foreign Language Center 
Attn: ATFL-VPT 

Presidio of Monterey, CA 93944-5006 

Digital Equipment Corporation 
12 Crosby Drive 
Bedford, MA 01730 

Digital Equipment Corporation 
146 Main Street 
Maynard, MA 01754 

Discworks, Inc. 
42 Crescfjnt Street 
Cambridge, W± U2138 

Diversified Data Resources, Inc. 
6609 Rosecroft Place 
Falls Church, VA 22043 

Eagle Technology, Inc. 
950 N. Orlando Avenue 
Winter Park, FL 32789 

Eastman Kodak Company 
343 State Street 
Rochester, NY 14650 

Education TURNKEY Systems 
256 N. Washington Street 
Falls Church, V/ 22046-4549 
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Educational Activities, Inc. 
P.O. Box 392 
Freeport, NY 11520 

Educational Data Systems 
22720 Michigan Avenue 
Dearborn, MI 48124 

Educational Testing Service 
Princeton, NJ 08541 

Edudisc, Inc. 
1400 Tyne Blvd. 
Nashville, TN 37215 

EECO, Incorporated 

1601 E. Chestnut 

Santa Ana, CA 92702-0659 

EER Systems 

3290 Progress Drive 

Orlando, PL 32826 

Essex Corporation 

1040 Woodcock Road - Suite 227 

Orlando, PL 32803 

European Institute of Education and 

Social Policy 
X Universite de Paris IX - Dauphine 
75116 Paris 
France 

Falcon Software 
P.O. Box 102 
Wentworth, NH 03282 

Federal Aeronautics Administration 

(FAA) Academy (ACC 912) 
Mike Monroney Aeronautical Center 
P.O. Box 25082 
Oklahoma City, OK 73125 

Federal Express 
2856 Directors Cove 
Memphis, TN 38131 



B-8 



Federal Law Enforcement Training Center 
Glenco, GA 31524 

First National Bank of Chicago 
1 N. State Street - Suite 0392 
Chicago, IL 60670 

FlightSafety International Communication 

Systems 
632C Bedford-Euless Road 
Hurst, TX 76053 

Florida Department of Health and 

Rehabilatative Services 
1311 Winewood Blvd. - Bldg 5, Room 131 
Tallahassee, FL 32301 

Florida State University 
College of Education 
215 Stone Building 
Tallahassee, FL 32306 

Ford Motor Company 
World Headquarters 
The American Road 
Dearborn, MI 48121-1899 

Ford Parts and Service 
Training Department 
3000 Sheaf fer Road 
Dearborn, MI 48121 

Forhan and Wakefield Group 
265 Post Road West 
Westpor^, CT 06880 

Fuld Institute for Technology in Nursing 

Education (FITNE) 
28 Station Street 
Athens, OH 45701 

Gant Associates 

222 Third Street - Suite 2242 

Cambridge, MA 02142 

General Electric Aircraft Engineering 
Educational Services Center 
Mail Drop El 99 
1 Neumann Way 
Cincinnati, Ohio 45215 
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George Washington University 
Department of Management Science 
Monroe Hall - Room 203 
2115 "G" Street, N,W. 
Washington, DC 20052 

Georgetown University School of Medicine 
TIME Project 
Kober-Cogan Building 
3800 Reservoir Road, N.W. 
Washington, DC 20007-2197 

Gery Associates 
5 Brookman Drive 
P.O. Box 851 

East Otis, MA 01029-0851 

Global Information Systems Technology, Inc. 
1800 Woodfield Drive 
Savoy, IL 61874-9505 

Golle & Holmes Custom Education 
1600 W. 82nd Street 
Minneapolis, MN 55431 

GPN 

P.O. Box 80669 
Lincoln, NE 68501 

GTE Corporation 
1 GTE Place 

Thousand Oaks, CA 91362-3811 

GTE Directories Corporation 
GTE Place West Airfield Drive 
P.O. Box 619810 
D/FW Airport, TX 75261 

Hane Industrial Training, Inc. 

120 South 7th Street 

P.O. Box 3165 

Terre Haute, IN 47803 
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Hansen Training Systems, Inc. 
1981 Abbotsford Drive 
Harrington, IL 60010-5560 

Harvard University Lav School 
Interactive Video Project 
Cambridge, MA 02138 

Harvard University 

Graduate School of Education 

Cambridge, MA 02138 

Harvard University 
Graduate School of Education 
Interactive Technology Office 
Cambridge, MA 02138 

Health EduTech 

7801 E. Bush Lake Road 

Minneapolis, MN 55435 

Hoffman Educational Systems 
1720 Flower Avenue 
Duarte, CA 91010 

Hope Reports, Inc. 
1600 Lyell Avenue 
Rochester, NY 14606 

Hoxvorth Blood Center 
3231 Burnet Avenue 
Cincinnati, OH 45267-0055 

Hughes Associates 

2730 University Blvd. West - Suite 902 
Wheaton, MD 20902 

Hughes Training Systems « Inc. 
2200 Arlington Downs Road 
Arlington, TX 76011 

IBM Corporation 

3301 Windy Ridge Parkway 

Marietta, OA 30067 

IBM Corporation 
PO.O. Box 2150 
Atlanta, GA 30055 
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IBM Corporation 
6905 Rockledge Crlve 
Bethesda, MD 20817 

IBM Corporation 

Thomas J. Watson Research Labor at o 
White Plains, NY 10604 

IBM - U.S. Education 
500 Columbus Road 
Thornwood, NY 10594 

ICON Associates, Inc. 

72 South La Grange Road, Suite 8 

La Grange, IL 60525 

Image Premasterlng Services, Ltd. 
1781 Prior Avenue North 
St. Paul, MN 55113 

Imedla International Inc. 
M.I.T. Branch 
P.O. Box 347 
Cambridge, MA 02139 

IMSATT Corporation 
Suite 101 

Falls Church, VA 22046 

Indiana University 
School of Education 
Education Technology Services 
Education 325 
Bloomlngton, IN 47401 

Industrial Training Corporation 
13515 Dulles Technology Drive 
Herndon, VA 22070 

Info-Disc Corporation 
4 Professional Drive 
Suite 134 

Galthersburg, MD 20879 
Info Tech 

4040 E. BlJou Street 
Colorado Springs, CO 80909 
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Information Technology Group 
9017 Shady Grove Court 
Gaithersburg, MD 20877 

The Institute of Electrical and 

Electronic Engineers 
445 Hoes Lane 
P.O. Box 1331 

Piscataway, NJ 08855-1331 

Institute for Interactive Learning 
4205 - 148th Avenue, N.E. - Suite 202 
Bellvue, WA 98007 

Instructional Design International 
1775 Church Street, N.W. 
Washington, DC 20036 

Instructional Science and Development 
4918 N. Harbor Drive 
Suite 201 

San Diego, CA 92106 
Intell 

201 Washington Road 
P.O. Box CN5325 
Princeton, NJ 08543 

Intelligent Images 
A Division of DaRox Corp. 
5510 Morehouse Drive 
San Diego, CA 92121 

Intellisance Cdex Corporation 
1855 Lundy Avenue 
San Jose, CA 95131 

Interactive Image Technologies 
49 Bathurst Street - Suite 401 
Toronto M5V 2P2 
Canada 

Interactive Instructional Systems, Inc. 
2 Gateway Center 
14 West 

Pittsburgh, PA 15222 
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Interactive Learning Syptems 
25 Sylvan Road South 
Westport, CT 06880 

Interactive Medical Cououanlcatlons 
100 Fifth Avenue 

Waltham, MA 02154 ^ 

Interactive Performance Systemd 
The John H. Harland Company 
(Interactive Financial Learning Systems) 
Box 105250 
Atlanta, GA 30348 

Interactive Technologies Corporation 
9625 Black Mountain Road 
San Diego, CA 92126 

Interactive Training Incorporated 
See Education Data Systems 

Interactive Training Systems 
See Spectr^^ Interactive 

Interactive Video Industry Association 
5929 Lee Highway 
Arlington, VA 22207 

Inter-ad, Inc. 
52 Marvay Circle 
Rochester, NY 14624 

Intercom, Inc. 
302 E. John Street 
Champaign, IL 61820 

» 

International Interactive Communications 

Society 
2120 Stelner Street 
San Francisco, CA 94115 

International Television Association 
6311 N. 0* Connor Road 
Irving, TX 75039 
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ISC Educational Systems, Inc. 

3700 Electronics Way 

P.O. Box 3040 

Lancaster, FA 17604-3040 

Xxlon, Inc. 

1335 N. Northlake Way 

Seattle, WA 98103 

Jarvls Associates 
10320 South 1435 West 
South v^ordan, UT 84065 

Jenson Publications Video Products Division 

2770 S. 171st Street 

New Berlin, WI 53151-0482 

Jeppesen Sanderson, Inc. 
55 Inverness Drive East 
Englewood, CO 80112-5498 

Kent State University 
College of Education 
Industrial Resources Center 
Kent, OH 44242 

Keyboard Productivity, Inc. 
6035 Bristol Parkway 
Culver City, CA 90230 

Klnton, Inc. 

5707 Seminary Road 

Bailey's Crossroads, VA 22041 

Leadership Studies, Inc. 
230 W. Third Avenue 
Escondldo, CA 92025 

Learncom 

A Division of Sandy Corporation 
215 First Street 
Cambridge, MA 02142 

Learning International 
P.O. Box 10211 
Stamford, CT 06904-9926 
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Learns tar. Inc. 

24333 Southfleld Road, Suite 205 
Southfield, MI 48075 

Llngu'^stlc Systems, Inc. 
116 Bishop Allen Drive 
Cambridge, MA 02139-0901 

Management Development Services, Inc. 
73 East Hanover Avenue 
Morris town, NJ 07960-2432 

Mankind Research Foundation 

1315 Apple Avenue 

Silver Spring, MD 20910-8776 

Marplex, Inc. 

71 M. Main Street 

Chagrin Falls, OH 44022 

Massachusetts Institute of Technology 
Audience Research Facility 
Cambridge, MA 02139 

Massachusetts Institute of Technology 
Departnent of Psychology 
Cambridge, MA 02139 

Massachusetts Mutual Life Insurance Company 
Springfield, MA 01111 

Mastertask Training Systems 
V-Tip, Inc. 
407 Green Street 
Rockford, IL 61105-0337 

MATROX Electronic Systems 
1055 St. Regis Blvd 
Dorval, Quebec 
Canada 

McDonnell Douglas Training 
8201 Greensboro Drive 
McLean, VA 22102 

MECC 

3490 Lexington Avenue, North 
St. Paul, MN 55126 
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The Media Exchange 
217 S. Payne Street 
Alexandria, VA 22314 

Media Learning Systems 
120 W. Colorado Blvd. 
Pasadena, CA 91105 

MedlcalDlsc Reporter 
6471 Merrltt Court 
Alexandria, VA 22312 

Meta Media 

20251 Century Blvd. 

Germantown, MD 20874 

Metropolitan Police Training Center 
PT8 

Training Support Services 

Peel Center 

Aerodrome Road 

London NW9 5JE 

U.K. 

Metrovest Communications Group 
P.O. Box 3308 
Framlngham, MA 01701 

Mlaml-Dade Community College 
Product Development and Distribution 
11011 S.W. 104th Street 
Miami, PL 33176 

Mlc*owonc, Inc. 

13911 Rldgedare Drive, #461 

Mlnnetonka, MN 55343 

MindBank, Inc. 

736 W. Ingomai Road - Suite 220 
P.O. Box 60 
Ingomar, FA 15127 

Mirror Systems, Inc. 
2067 Massachusetts Avenue 
Cambridge, MA 02140 
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MITEC 

11767 Bonlta Avenue 
Ovlngs Mills, MD 21117 

National Cryptologlc School 
Ft. Meade, MD 20755 

Nat:'onal Interactive Video Center 
24-32 Stephenson Way 
London NWl 2HD 
U.K. 

National Library of Medicine 
Lister HiU Center 
8600 Rockvllle Pike 
Bethesda, MD 20894 

National School Board Association 
1600 Duke Street 
Alexandria, VA 22314 

National Science Center for 
Communications Electronics 
P.O. Box 1648 
Augusta, GA 30903 

National Security Agency 
Attn: E33 

Ft. Meade, MD 20785-6000 

Naval Training Systems Center 
Code 10 

Orlando, FL 32826 

Naval Training Systems Center 
Code 113 

Orlando, FL 32826-3224 

Navy Personnel Research and Dev. Center 
San Diego, CA 92152 

Nebraska Interactive Video 
P.O. Box 4707 
Lincoln, NE 68504 
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Nebraska Videodisc Group 
P.O. Box 8311 
Lincoln, NE 68501 

New Mexico State Unversity 
Agricultural Information Center 
Las Cruces, NM 88003 

New Mexico State Univers:*.ty 
College of Education 
Las Cruces, NM 88003 

Newport News Ship Building 

2711 S. Jefferson Davis Highway - Suite 1100 

Arlington, VA 22202 

North Commtinications 
3030 Pennsylvania Avenue 
Santa Monica, CA 90404 

Northrop Corporation - Aircraft Division 
1 Northrop Avenue - 1560/40 
Hawthorne, CA 90250-3277 

Office of Technology Assessment /SET 
U.S. Congress 

Washington, DC 20510-8025 

Omni Com Associates 
407 Coddington Road 
Ithicd, NY 14850 

Online Computer Systems, Inc. 
20251 Century Blvd. 
Germantown, MD 20874 

Optical Data Corporation 
30 Technology Drive 
P.O. Box 4919 
Warren, NJ 07060 

Oregon State University 
Media Laboratory 
School of Education 
Corvallis, OR 97331 
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Paperback Video Publishing, Inc. 

29 Broadway 

New York, NY 10006 

Pennsylvania State Unviversity 
Instructional Systems Department 
246 Chambers 

University Park, PA 16802 
Perceptronics, Inc. 

Training and Simulation Systems Division 
2122 Ervin Street 
Woodland Hills, CA 91367 

Performax 

1720 Post Road East 
Westport, CT 06880 

Person-System Integration 
2401 Huntington Avenue 
Alexandria, VA 22303-1531 

Phoenix Interactive Design 
470 Ridout Street North 
London ONT 2P7 
Canada 

Pioneer 

1058 E. 230th Street 
Carson, CA 90745 

Pioneer 

600 E* Crescent Avenue 

Upper Saddle River, NJ 07458 

PJV Inc. 

See the Alive Center 

Pratt & Whitney 

400 Main Street - MS 117-34 

East Hartford, CT 06108 

Pratt & Whitney 
P.O. Box 109600 
West Palm Beach, FL 33410 
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Professional Training Systems, Inc- 
400 Colony Square - Suite 1525 
1201 Peachtree Street, N*E, 
Atlanta, GA 30361 

Rand Corporation 
1700 Main Street 
P.O. Box 2138 

Santa Monica, CA 90406-2138 

Regency Systems, Inc« 
3200 Farber Drive 
Champaign, IL 61821-3578 

RJO Enterprise 
4550 Forbes Blvd. 
Lanham, MD 2 J706 

Reynolds and Reynolds 
800 Germantovn Street 
Dayton, OH 45407 

San Diego State University 
Department of Educational Technology 
San Diego. CA 92182-0311 

San Jose State University 

Department of Instructional Technology 

San Jose, CA 95192 

Sandy Corporation 

1500 W, Big Beaver Road 

Troy, MI 48084 

Science Research Associates, Inc. 
155 North Wacker Drive 
Ch' cago, IL 60606 

Scientific Systems Incorporated 
1 Alevife Place 
Cambridge, MA 02140 

Stanton C. Selbst, Inc. 

7-11 S. Broadway 

White Plains, NY 10601 
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Seville Training 
A Division of FlightSafety 
2445 Gateway Drive 
Irving, TX 75063 

Sony Corporation of America 
Intelligent Systems Group 
Sony Drive 

Park Ridge, NJ 07656 

Source Interactive 
llSl South Parker Road 
Denver, CO 80231 

Southern Oregon State College 
Education Department 
Ashland, OR 97520 

South-Western Publishing Company 
5101 Madison Road 
Cincinnati, OH 45227 

Spectrum Interactive (ITS, Inc.) 
9 Oak Park Drlvo 
Bedford, MA 01730 

Stanford University 
ACIS/IRIS 

Swee: Hall - 3rd Floor 
Stanford, CA 94305-3091 

State University of New York - Binghamton 
Department of Mechanical Engineering 
Binghamton, NY 13901 

Statistica, Inc. 
One Metro Square 
51 Monroe Street 
Rockvllle, MD 20850 

Synergistic Educational Technology 

Systems, Inc. 
4405 Vineland Road 
Suite C4 

Orlando, FL 32811 
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Syracuse University 

Instructional Design, Development and 

Evaluation 
330 Huntington Hall 
Syracuse, NY 13244 

SYSCON 

12424 Research Parkway 
Suite 390 

Orlando, FL 32826 
Systems Impact 

4400 McArthur Blvd., N.W. - Suite 203 
Washington, DC 20007 

Tactical Air Command 
U.S. Air Force 

4400th Maintenance Training Flight (TAC) 
Hill Air Force Base, UT 84056-5000 

TAPPI 

Technology Park 
Atlanta, OA 30348 

TCT Technical Training, Inc. 
599 N. Mathilda Avenue 
Sunnyvale, CA 94086 

TEAC Corporation of America 
7733 Telegraph 
Montebello, CA 90640 

Tel-A-Train, Tnc. 
309 N. Market Street 
Chattanooga, IN 37405 

Tennessee Valley Authority 
Skills Development 
601 W. Summit Hill Drive 
1B31 Old City Hall Building 
Knoxville, IN 37902 

Texas School Board Association 
Texas Learning Technology Group 
P.O. Box 2947 
Austin, TX 78769-2947 
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3h Optical Recording 
3M Center 223-5S-01 
St. Paul, MN 55144 

T/K International, Incorporated 
663A Market Hill Road 
Vancouver, BC V5Z 4B 
Canada 

TPC Training Systems 
Div« of Telemedla, Inc* 
310 S. Michigan Avenue 
Chicago, IL 60604 

TRADOC 

ATRC-WGB 

TRAC-WSMR 

White Sands Missile Range, NM 88002-5502 

Training and Development Resources 
104 E. Main Street 
Durham, NC 27707 

The Training Group, Inc, 
#202, 4220-98 Street 
Edmonton, Alberta T6E 6A1 
Canada 

Training and Performance Data Center 
3280 Progress Drive 
Orlando, FL 32826 

Training Solutions, Inc. 
2314 Lincoln Park West 
Chicago, IL 60614 

Training Solutions, Inc. 
4647 Willis - #106 
Sherman Oaks, CA 91403 

Uniformed Services University of 

the Health Sciences 
Center for Interactive Media in Medicine 
Department of Military Medicine 
4301 Jones Bridge Road 
Bethesda, MD 20814 
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UNISYS Education Center 
P.O. Box 1110 
Princeton, NJ 08543 

United Auto Workers/General Motors 

Health and Safety Training Center 
29815 John R 

Madison Heights, MI A8071 

United States Air Force Academy 
Department of Foreign Languages 
USAF/DFF 

Colorado Springs, CO 80840-5701 

United States Army Training Support Center 
Ft. Eustls, VA 23604-5168 

United States Department of Agriculture 

3322-S 

USDA-ES-CIT 

Washington, DC 20025-0900 

United States Military Academy 
Department of Foreign Languages 
West Point, NT 10996 

United States Naval Academy 
Department of Language Studies 
Annapolis, MD 21402-5030 

University of Akron 

Center for Computer-Based Education 

Carroll Hall 

Akron, OH 44325 

Unlveislty of Alberta 
Instructional Technology Center 
Bll? Education North 
Alberta T6G 2G5 
Canada 

University of California - Davis 
Department of Education 
Davis, CA 95616 
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